-- Logs begin at Thu 2022-06-16 17:45:57 UTC, end at Thu 2022-06-16 19:17:44 UTC. -- Jun 16 18:36:16.472678 dv-cinder-846171-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 18:36:16.481754 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 18:36:16.483773 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 18:36:16 2022] *** Jun 16 18:36:16.483773 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 18:36:16.483773 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 18:36:16.483773 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: nodename: dv-cinder-846171-1-iscsi Jun 16 18:36:16.483773 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: machine: x86_64 Jun 16 18:36:16.483773 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: clock source: unix Jun 16 18:36:16.483773 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: pcre jit disabled Jun 16 18:36:16.483773 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: detected number of CPU cores: 4 Jun 16 18:36:16.483773 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: current working directory: / Jun 16 18:36:16.483773 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: detected binary path: /usr/bin/uwsgi-core Jun 16 18:36:16.483773 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: your processes number limit is 47680 Jun 16 18:36:16.484545 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: your memory page size is 4096 bytes Jun 16 18:36:16.484545 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: detected max file descriptor number: 2048 Jun 16 18:36:16.484545 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: lock engine: pthread robust mutexes Jun 16 18:36:16.484545 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: thunder lock: enabled Jun 16 18:36:16.484545 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 18:36:16.484545 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 18:36:16.527607 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: Python main interpreter initialized at 0x55b18b8fc9c0 Jun 16 18:36:16.527607 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: python threads support enabled Jun 16 18:36:16.527607 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: your server socket listen backlog is limited to 100 connections Jun 16 18:36:16.527607 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: your mercy for graceful operations on workers is 90 seconds Jun 16 18:36:16.527941 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 18:36:16.528018 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: *** Operational MODE: preforking *** Jun 16 18:36:16.528282 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: *** uWSGI is running in multiple interpreter mode *** Jun 16 18:36:16.528282 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: spawned uWSGI master process (pid: 92256) Jun 16 18:36:16.528474 dv-cinder-846171-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 18:36:16.533284 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: spawned uWSGI worker 1 (pid: 92269, cores: 1) Jun 16 18:36:16.533284 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: spawned uWSGI worker 2 (pid: 92270, cores: 1) Jun 16 18:36:16.533284 dv-cinder-846171-1-iscsi devstack@keystone.service[92256]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 18:36:22.034573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:36:22.034573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: from cryptography.utils import int_from_bytes Jun 16 18:36:22.035369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:36:22.035369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: from cryptography.utils import int_from_bytes Jun 16 18:36:22.035810 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:36:22.035810 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: from cryptography.utils import int_from_bytes Jun 16 18:36:22.035943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 18:36:22.035943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: from cryptography.utils import int_from_bytes Jun 16 18:36:22.085032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 18:36:22.085215 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 18:36:22.085435 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] command line args: [] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 18:36:22.085717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 18:36:22.085901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ================================================================================ {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 18:36:22.086162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] admin_token = **** {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.086415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] config_dir = [] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.086607 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.086795 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] config_source = [] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.086983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] debug = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.087188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.087691 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.087785 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.087999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.088258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.088494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] list_limit = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.088679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] log_config_append = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.088861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.089043 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] log_dir = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.089224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] log_file = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.089488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.089743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.090014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.090251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.090409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 18:36:22.090489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.090612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 18:36:22.090803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] command line args: [] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 18:36:22.090876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.091061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 18:36:22.091141 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.091259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ================================================================================ {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 18:36:22.091338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.091488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] admin_token = **** {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.091572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.091740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] config_dir = [] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.091822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.091931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.092007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.092117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] config_source = [] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.092286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] debug = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.092502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.092745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.092926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.093186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.093469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.093705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] list_limit = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.093888 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] log_config_append = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.094071 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.094250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] log_dir = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.094430 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] log_file = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.094671 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.094948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.095731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.095989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.096171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.096351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.096603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.096866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.097094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.097366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.097606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.097847 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.098099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.178009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.178320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.178639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.179182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.81/identity {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.179382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] publish_errors = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.179638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.179890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.180086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.180357 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.180614 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.180868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] standard_threads = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.181117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.181405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.181596 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.181781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] use_journal = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.181967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] use_json = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.182149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] use_stderr = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.182333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] use_syslog = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.182579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.182902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.183126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.183368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.183620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.183884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.184025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.184136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.184376 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.184454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.184634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.184895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.184975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.81/identity {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.185100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] publish_errors = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.185183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.185385 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.185463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.185639 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.185724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.185831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.185906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.186102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.186177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.186358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.186436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.186612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] standard_threads = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.186689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.186859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.186940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.187122 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.187202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.187307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.187384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.187492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] use_journal = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.187624 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.187710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] use_json = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.187843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] use_stderr = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.187919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.188025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] use_syslog = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.188144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.188256 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 18:36:22.188374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.188529 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.188639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.188830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.188929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.189103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.189182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.189381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.189471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.189638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.189720 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.189884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.189960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.190140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.190218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.190386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.190512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.190622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.190698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.190860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.190936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.191115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.191191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.191363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.191447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.191606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.191683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.191849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.191931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.192091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.192172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.192336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.192411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.192579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.192655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.192817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.192899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.193054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.193129 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.193311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.193390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.193551 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.193656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.193794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.193895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.194054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.194155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.194294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.194395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.194537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.194638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.194774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.194881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.195029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.195140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.195256 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.195382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.195523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.195608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.195731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.195806 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.195985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.196070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.196205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.196329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.196442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.196534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.196681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.196756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.196924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.197016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.197160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.197291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.197441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.197526 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.197681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.197765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.197930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.198055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.198165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.198242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.198408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.198487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.198647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.198734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.198887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.199001 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.199121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.199197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.199362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.199448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.199617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.199693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.199857 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.199945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.200102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.200190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.200341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.200451 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.200584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.200660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.200823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.200906 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.201065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.201177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.201327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.201406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.201575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.201664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.201816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.201892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.202058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.202134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.202300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.202383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.202541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.202620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.202780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.202856 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.203020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.203112 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.203261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.203338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.203506 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.203581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.203745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.203895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.204004 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.204086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.204247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.204322 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.204513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.204604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.204756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.204833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.204997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.205072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.205258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.205336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.205493 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.205572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.205731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.205815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.205970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.206045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.206212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.206293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.206448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.206525 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.206695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.206771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.206935 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.207013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.207179 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.207260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.207421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.207503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.207670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.207747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.207915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.207991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.208161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.208248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.208402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.208480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] auth.external = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.208637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.208713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.208878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.208953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.209149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.209227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.209418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] auth.password = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.209494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.209656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] auth.token = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.209732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.209899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.209976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.210139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.210215 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.210380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.210455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.210622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.210698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.210862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.210938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.211100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.211175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.211337 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.211414 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.211573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.211649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] credential.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.211815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.211901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.212051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.212128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.212300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.212384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.212541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.212616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.212794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.212870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.213033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.213117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.213316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.213412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.213580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.213661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.213824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.213903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.214062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] auth.external = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.214138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.214301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.214378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.214609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.214688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.214848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.214924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.215086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] auth.password = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.215162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.215323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] auth.token = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.215406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.215573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.215656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] federation.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.215818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.215894 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.216070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.216148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.216308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.216391 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.216548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.216632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.216803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.216881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.217040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.217117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.217313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] credential.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.217539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.217626 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.217702 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.217818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.217915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.218078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.218163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.218324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.218401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.218577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.218652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.218817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.218900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.219075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.219151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.219317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.219393 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.219559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.219643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.219813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.219890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.220587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.220670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.220833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.220918 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.221092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.221169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.221376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.221466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.221619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.221704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.221857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.221934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] federation.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.222095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.222172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.222350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.222435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.222605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.222682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.222856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.222931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.223115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.223199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.223362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.223440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.223627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.223703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.223872 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.223956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.224115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.224190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.224352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.224429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.224597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.224681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.224834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.224911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.225075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.225151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.225344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.225438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.225590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.225674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.225835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.225912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.226072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.226156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.226322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.226398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.226568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.226645 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.226806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.226890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.227061 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.227138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.227299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.227375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.227537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.227615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.227774 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.227850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.228007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.228082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.228251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.228334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.228499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.228576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.228747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.228824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.228992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.229069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.229248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.229330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.229519 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.229596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.229764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.229841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.230000 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.230327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.230491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.230568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.230728 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.230805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.230973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.231050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.231219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.231296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.231458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.231535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.231699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.231776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.231935 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.232012 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.232180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.232259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.232434 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.232509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.232672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.232748 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.232925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.233002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.233177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.233294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.233438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.233515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.233682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.233760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.233916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.233991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.234153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.234229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.234394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.234472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.234635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.234712 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.234871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.234948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.235113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.235190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.235366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.235441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.235663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.236268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.236431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.236519 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.236670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.236748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.236906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.236984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.237159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.237256 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.237425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.237502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.237722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.237809 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.237899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.237972 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.238131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.238209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.238380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.238457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.238639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.238716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.238876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.238954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.239135 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.239212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.239371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.239448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.239622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.239699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.239863 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.239939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.240101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.240180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.240341 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.240417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.240581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.240657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.240826 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.240904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.241060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.241136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.241325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.241406 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.241575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] resource.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.241654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.241828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.241904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.242081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.242229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.242354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.242429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.242560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.242637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.242814 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.242900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.243055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.243131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.243297 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.243377 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.243540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.243624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.243781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.243856 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.244020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] role.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.244096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.244274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.244359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] role.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.244508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.244584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.244762 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.244839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.244996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.245080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.245285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.245368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.245513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.245590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.245757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.245846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.246003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.246080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.246254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.246329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.246501 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.246585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.246738 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.246813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.246980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.247055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.247226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] resource.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.247309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.247478 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.247555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.247730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.247805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.247976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.248149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.248227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.248395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.248471 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.248654 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.248737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.248909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.248987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.249165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.249262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.249441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.249518 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.249682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.249760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] role.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.249936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] role.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.250011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.250176 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.250253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.250428 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.250505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.250667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.250743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.250905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.250982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.251144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.251222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.251432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.251520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.251684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.251769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.251937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.252013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.252183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.252260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.252421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.252508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.252657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.252733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.252900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] token.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.252977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.253146 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.253231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.253436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.253514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.253682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.253831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.253948 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.254073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.254205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.254330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.254449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.254588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.254700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.254847 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.254950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.255104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.255206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.255346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.255450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.255609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.255687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.255871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.255948 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.256117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.256193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.256362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.256440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.256601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.256677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.256838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.256915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.257098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.257177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.257396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.257476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.connection = **** {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.257653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.257739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.257896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.257982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.258141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.258218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.258378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.258455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.258621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] token.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.258706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.258863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.258940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.259109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.259185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.259350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.259433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.259600 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.259676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.259842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.259919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.260087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.260171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.260327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.260403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.260576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.260652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.260816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.260899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.261063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.261139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.261331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.261413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.261572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.261704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.261812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.261889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.261985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 18:36:22.262063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.262291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.262540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.262842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.263111 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.connection = **** {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.263363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.263620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.263866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.264114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.264358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.264602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.264845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.265089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.265376 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.265622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.265874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.266115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.266377 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.266619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.266890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.267131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.267381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.267623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 18:36:22.267797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92270) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 18:36:22.291805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.291986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.292217 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.292578 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.292813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.293042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.301114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.302296 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.328823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.329064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.329771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.330007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.382980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.383324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.431705 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.431941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.432072 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.432314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.498005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.498234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.498355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.498466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.498590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.498692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.498817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.498899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.499038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.499118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.499257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.499474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.522464 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.522695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.522783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.522921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.523005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.523140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.523222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.523355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.523437 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.523567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.523651 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.523781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.523863 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.523996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.535375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.535618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.548074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.548315 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.548401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.548545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.548629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.548765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.548846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.548980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.549061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.549193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.549295 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.549433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.549515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.549649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92270) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 18:36:22.566651 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.566862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:36:22.567176 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.567259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.567390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:36:22.567485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:36:22.567661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.567742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.567913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 18:36:22.567994 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.568169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.568257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.568426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.568507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.568692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.568953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.579292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.580111 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.580351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.580434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.580631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.580711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.580902 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.581184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.588889 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.589443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.592209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.592489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.592727 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.592810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.593008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.593089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.593313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.593399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.593592 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.593673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.593861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.593954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.594120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.594201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.594388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.594467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.594660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.594748 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.594926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.595007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.595775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.596046 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.622147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.622732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.623763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.624044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.624342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.624625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.629375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.629776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.629949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.630060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.630344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.630628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.637468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.637751 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.637909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.638193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.642772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.643056 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.643219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.643338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.643505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.643618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.643787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.643893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.644067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.644167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.644341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.644430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.644615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.644695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.644884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.645151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.663957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.664238 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.664451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.664732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.668786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.669344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.672492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.672893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.673018 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.673424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.677984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.678420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.681751 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.682063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.685411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.685711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.685807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.685997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.686087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.686279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.686361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.686556 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.686647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.686829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.686919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.687107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.708729 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.709014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.709097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.709305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.709406 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.709586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.709668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.709851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.717500 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.717769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.719204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.719475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.719558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.719771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.723267 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.723550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.723634 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.723834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.723915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.724117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.724209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.724393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.724474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.724668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.738063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.738233 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.738344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.738514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.738635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.738795 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.738914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.739078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.739286 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.739453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.739562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.739787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.739875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.740105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.757715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.758025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.760979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.761290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.764315 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.764593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.766082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.766360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.768266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.768544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.768628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.768823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.768904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.769115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.776134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.776411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.777787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.778064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.781013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.781308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.781447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.781606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.781720 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.781997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.782087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.782280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.791755 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.792049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.792175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.792327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.792461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.792741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.800286 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.800515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.800724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92270) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 18:36:22.800808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.801004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.801085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.801303 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.801388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.801583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.801708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.801859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.801986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.802247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.802336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.802524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.802605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.802800 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.802881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.803071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.803154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.803341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.803422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.803606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.803686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.803869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.803951 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.804134 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.804394 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92270) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 18:36:22.842450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b18b8fc9c0 pid: 92269 (default app) Jun 16 18:36:22.843023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b18b8fc9c0 pid: 92270 (default app) Jun 16 18:36:22.845971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da94df-3c46-498c-a1f9-2c565ed3b4f5 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:22.846167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da94df-3c46-498c-a1f9-2c565ed3b4f5 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:22.846452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da94df-3c46-498c-a1f9-2c565ed3b4f5 None None] PATH_INFO: `/v3` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:22.850734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1/1] 10.222.0.81 () {50 vars in 791 bytes} [Thu Jun 16 18:36:22 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:31.532616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cf08b-8ee9-409f-843c-688977f56f25 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:31.533164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cf08b-8ee9-409f-843c-688977f56f25 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:31.533164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240cf08b-8ee9-409f-843c-688977f56f25 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:31.538090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1/2] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:36:31.543162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8d154-4a8b-471f-ac2c-8fd43541f67e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:31.543589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8d154-4a8b-471f-ac2c-8fd43541f67e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:31.543715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8d154-4a8b-471f-ac2c-8fd43541f67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:31.612908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: INFO dbcounter [None req-30c8d154-4a8b-471f-ac2c-8fd43541f67e None None] Registered counter for database keystone Jun 16 18:36:31.633379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG oslo_db.sqlalchemy.engines [None req-30c8d154-4a8b-471f-ac2c-8fd43541f67e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92269) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 18:36:31.639465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writer thread running {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 18:36:31.813393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG passlib.handlers.bcrypt [None req-30c8d154-4a8b-471f-ac2c-8fd43541f67e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92269) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 18:36:31.813818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG passlib.handlers.bcrypt [None req-30c8d154-4a8b-471f-ac2c-8fd43541f67e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92269) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 18:36:31.868173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-30c8d154-4a8b-471f-ac2c-8fd43541f67e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:31.869816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-30c8d154-4a8b-471f-ac2c-8fd43541f67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:31.991650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 2/3] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 18:36:32.131615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cf08b-8ee9-409f-843c-688977f56f25 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:32.202857 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: INFO dbcounter [None req-240cf08b-8ee9-409f-843c-688977f56f25 None None] Registered counter for database keystone Jun 16 18:36:32.221543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG oslo_db.sqlalchemy.engines [None req-240cf08b-8ee9-409f-843c-688977f56f25 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92270) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 18:36:32.227728 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writer thread running {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 18:36:32.346201 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240cf08b-8ee9-409f-843c-688977f56f25 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:32.350456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62051d7-efca-4b28-907e-04b49b97618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:32.351112 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62051d7-efca-4b28-907e-04b49b97618f admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:32.351396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62051d7-efca-4b28-907e-04b49b97618f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:32.351673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62051d7-efca-4b28-907e-04b49b97618f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:32.360104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62051d7-efca-4b28-907e-04b49b97618f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:32.412643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b62051d7-efca-4b28-907e-04b49b97618f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 18:36:32.412997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62051d7-efca-4b28-907e-04b49b97618f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:32.422773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 2/4] 10.222.0.81 () {58 vars in 1324 bytes} [Thu Jun 16 18:36:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:36:32.429165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8d154-4a8b-471f-ac2c-8fd43541f67e None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:32.445173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8d154-4a8b-471f-ac2c-8fd43541f67e None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:32.449629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0f6ab-9c03-45a3-a3ac-9e1fab2b3ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:32.450723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0f6ab-9c03-45a3-a3ac-9e1fab2b3ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:32.451009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0f6ab-9c03-45a3-a3ac-9e1fab2b3ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:32.451300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0f6ab-9c03-45a3-a3ac-9e1fab2b3ba3 admin admin] PATH_INFO: `/v3/projects/3b167122bd2b4c3cba143eecf3576355` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:32.452782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0f6ab-9c03-45a3-a3ac-9e1fab2b3ba3 admin admin] RBAC: Authorizing `identity:get_project(project_id=3b167122bd2b4c3cba143eecf3576355)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:32.503771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5ce0f6ab-9c03-45a3-a3ac-9e1fab2b3ba3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 18:36:32.504110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce0f6ab-9c03-45a3-a3ac-9e1fab2b3ba3 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:32.508767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 3/5] 10.222.0.81 () {56 vars in 1207 bytes} [Thu Jun 16 18:36:32 2022] GET /identity/v3/projects/3b167122bd2b4c3cba143eecf3576355 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:34.615133 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8f97d-2a01-4cd0-beb5-c2c1f072b88a None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:34.615632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8f97d-2a01-4cd0-beb5-c2c1f072b88a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:34.615632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b8f97d-2a01-4cd0-beb5-c2c1f072b88a None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:34.619751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 3/6] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:36:34.625175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94946b61-deb9-4b7b-b870-abc378f79052 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:34.625658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94946b61-deb9-4b7b-b870-abc378f79052 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:34.625749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94946b61-deb9-4b7b-b870-abc378f79052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:34.644607 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-94946b61-deb9-4b7b-b870-abc378f79052 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:34.646178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-94946b61-deb9-4b7b-b870-abc378f79052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:34.654078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 4/7] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 18:36:34.676504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8f97d-2a01-4cd0-beb5-c2c1f072b88a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:34.688419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b8f97d-2a01-4cd0-beb5-c2c1f072b88a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:34.692658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d14f28-c6d1-4082-baac-e004b33086eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:34.693309 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14f28-c6d1-4082-baac-e004b33086eb admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:34.693602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14f28-c6d1-4082-baac-e004b33086eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:34.693884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d14f28-c6d1-4082-baac-e004b33086eb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:34.701441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d14f28-c6d1-4082-baac-e004b33086eb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:34.703237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d14f28-c6d1-4082-baac-e004b33086eb admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:34.712299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 4/8] 10.222.0.81 () {58 vars in 1324 bytes} [Thu Jun 16 18:36:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:36:34.717450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94946b61-deb9-4b7b-b870-abc378f79052 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:34.726825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94946b61-deb9-4b7b-b870-abc378f79052 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:34.731135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc0496-ebde-4dc0-a617-29e02e9de145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:34.732234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc0496-ebde-4dc0-a617-29e02e9de145 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:34.732514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc0496-ebde-4dc0-a617-29e02e9de145 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:34.732785 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc0496-ebde-4dc0-a617-29e02e9de145 admin admin] PATH_INFO: `/v3/users/88fee257b59d48f2b418ffa9e86b6f8e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:34.734170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc0496-ebde-4dc0-a617-29e02e9de145 admin admin] RBAC: Authorizing `identity:get_user(user_id=88fee257b59d48f2b418ffa9e86b6f8e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:34.735989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc0496-ebde-4dc0-a617-29e02e9de145 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:34.737559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 5/9] 10.222.0.81 () {56 vars in 1198 bytes} [Thu Jun 16 18:36:34 2022] GET /identity/v3/users/88fee257b59d48f2b418ffa9e86b6f8e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:36.916517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458921d0-2888-43af-8f2d-821c882f4b09 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:36.917053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458921d0-2888-43af-8f2d-821c882f4b09 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:36.917053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458921d0-2888-43af-8f2d-821c882f4b09 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:36.918621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 5/10] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:36.923618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc3cea-1e64-44eb-bc6e-f5baadf976a2 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:36.924003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc3cea-1e64-44eb-bc6e-f5baadf976a2 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:36.924191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc3cea-1e64-44eb-bc6e-f5baadf976a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:36.925631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f9ae5-7a6e-449d-afd9-6f614e505dab None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:36.925925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f9ae5-7a6e-449d-afd9-6f614e505dab None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:36.926207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f9ae5-7a6e-449d-afd9-6f614e505dab None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:36.927439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 6/11] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:36.932351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926558e-15e8-459a-b6d9-15731ef2d963 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:36.932646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926558e-15e8-459a-b6d9-15731ef2d963 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:36.932925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926558e-15e8-459a-b6d9-15731ef2d963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:36.941867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d1bc3cea-1e64-44eb-bc6e-f5baadf976a2 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:36.943099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d1bc3cea-1e64-44eb-bc6e-f5baadf976a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:36.973188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG passlib.handlers.bcrypt [None req-2926558e-15e8-459a-b6d9-15731ef2d963 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92270) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 18:36:36.973642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG passlib.handlers.bcrypt [None req-2926558e-15e8-459a-b6d9-15731ef2d963 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92270) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 18:36:37.019131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-2926558e-15e8-459a-b6d9-15731ef2d963 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:37.020732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-2926558e-15e8-459a-b6d9-15731ef2d963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:37.022177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 6/12] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:37.046008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc3cea-1e64-44eb-bc6e-f5baadf976a2 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:37.069664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:37.122379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 7/13] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:36:37.144896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926558e-15e8-459a-b6d9-15731ef2d963 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:37.149231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc3cea-1e64-44eb-bc6e-f5baadf976a2 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:37.185032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f04881-19d8-412c-b8c6-3c2815059598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:37.186330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f04881-19d8-412c-b8c6-3c2815059598 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:37.186813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f04881-19d8-412c-b8c6-3c2815059598 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:37.187246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f04881-19d8-412c-b8c6-3c2815059598 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:37.199692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f04881-19d8-412c-b8c6-3c2815059598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-93f04881-19d8-412c-b8c6-3c2815059598 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:37.210937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:37.211941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:36:37.212760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 7/14] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:36:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:36:37.217923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f04881-19d8-412c-b8c6-3c2815059598 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:37.247481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926558e-15e8-459a-b6d9-15731ef2d963 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:37.249758 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:37.283549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d97fe9-ce17-492d-80c8-d046439c8894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:37.284959 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d97fe9-ce17-492d-80c8-d046439c8894 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:37.285261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d97fe9-ce17-492d-80c8-d046439c8894 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:37.285556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d97fe9-ce17-492d-80c8-d046439c8894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:37.298199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:37.315992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f04881-19d8-412c-b8c6-3c2815059598 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:37.350772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8e63f-0fe5-442e-b783-75d21aa45b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:37.351434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8e63f-0fe5-442e-b783-75d21aa45b53 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:37.351718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8e63f-0fe5-442e-b783-75d21aa45b53 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:37.351991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8e63f-0fe5-442e-b783-75d21aa45b53 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:37.352455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d97fe9-ce17-492d-80c8-d046439c8894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:37.352728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8e63f-0fe5-442e-b783-75d21aa45b53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:37.353035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8e63f-0fe5-442e-b783-75d21aa45b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:37.354015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d97fe9-ce17-492d-80c8-d046439c8894 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:37.354462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8e63f-0fe5-442e-b783-75d21aa45b53 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:37.366983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 8/15] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:36:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:36:37.372369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8e63f-0fe5-442e-b783-75d21aa45b53 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:37.453060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 8/16] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:36:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:36:37.459022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d97fe9-ce17-492d-80c8-d046439c8894 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:37.461656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:37.471216 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8e63f-0fe5-442e-b783-75d21aa45b53 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:37.504835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7441eb-b870-4254-959d-38c38fb582ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:37.505739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7441eb-b870-4254-959d-38c38fb582ee None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:37.506019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7441eb-b870-4254-959d-38c38fb582ee None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:37.506290 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7441eb-b870-4254-959d-38c38fb582ee None admin] PATH_INFO: `/v3/users/88fee257b59d48f2b418ffa9e86b6f8e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:37.507633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7441eb-b870-4254-959d-38c38fb582ee None admin] RBAC: Authorizing `identity:get_user(user_id=88fee257b59d48f2b418ffa9e86b6f8e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:37.508927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7441eb-b870-4254-959d-38c38fb582ee None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:37.510704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 9/17] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:37 2022] GET /identity/v3/users/88fee257b59d48f2b418ffa9e86b6f8e => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:36:37.515732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7441eb-b870-4254-959d-38c38fb582ee None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:37.530081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:37.556473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d97fe9-ce17-492d-80c8-d046439c8894 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:37.590085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ec9c9-133c-42f2-825c-595f3d32137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:37.590877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ec9c9-133c-42f2-825c-595f3d32137e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:37.591164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ec9c9-133c-42f2-825c-595f3d32137e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:37.591450 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ec9c9-133c-42f2-825c-595f3d32137e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:37.610133 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ec9c9-133c-42f2-825c-595f3d32137e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:37.611609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ec9c9-133c-42f2-825c-595f3d32137e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:37.613015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7441eb-b870-4254-959d-38c38fb582ee None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-237ec9c9-133c-42f2-825c-595f3d32137e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:36:37.632222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:37.633060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:37.633920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:36:37.633920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:36:37.633920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:36:37.633920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:36:37.633920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 9/18] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:36:37.639084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ec9c9-133c-42f2-825c-595f3d32137e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:37.646306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e5c1f-9b7b-4691-bc57-7ab8e2e503b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:37.647105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e5c1f-9b7b-4691-bc57-7ab8e2e503b1 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:37.647387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e5c1f-9b7b-4691-bc57-7ab8e2e503b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:37.647662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e5c1f-9b7b-4691-bc57-7ab8e2e503b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:37.648831 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e5c1f-9b7b-4691-bc57-7ab8e2e503b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:37.650342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e5c1f-9b7b-4691-bc57-7ab8e2e503b1 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:37.651929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 10/19] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:36:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:36:37.657763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e5c1f-9b7b-4691-bc57-7ab8e2e503b1 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:37.694739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:37.735830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:37.738996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ec9c9-133c-42f2-825c-595f3d32137e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:37.754111 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e5c1f-9b7b-4691-bc57-7ab8e2e503b1 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:37.772253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6f663-8247-4947-93bd-b18b93440002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:37.772920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6f663-8247-4947-93bd-b18b93440002 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:37.773210 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6f663-8247-4947-93bd-b18b93440002 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:37.773520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6f663-8247-4947-93bd-b18b93440002 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:37.774241 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6f663-8247-4947-93bd-b18b93440002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:37.774554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6f663-8247-4947-93bd-b18b93440002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:37.775948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6f663-8247-4947-93bd-b18b93440002 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:37.788858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee564bf-c618-4d64-97cf-a4909c8fd774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:37.789544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee564bf-c618-4d64-97cf-a4909c8fd774 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:37.789833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee564bf-c618-4d64-97cf-a4909c8fd774 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:37.790114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee564bf-c618-4d64-97cf-a4909c8fd774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:37.790935 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee564bf-c618-4d64-97cf-a4909c8fd774 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.domain.id=default, user.id=88fee257b59d48f2b418ffa9e86b6f8e) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:37.791247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee564bf-c618-4d64-97cf-a4909c8fd774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:37.792694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee564bf-c618-4d64-97cf-a4909c8fd774 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:37.793815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 10/20] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:37.799470 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 11/21] 10.222.0.81 () {56 vars in 1322 bytes} [Thu Jun 16 18:36:37 2022] GET /identity/v3/role_assignments?user.id=88fee257b59d48f2b418ffa9e86b6f8e&scope.domain.id=default&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:36:39.973693 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2c2a7-639c-4891-8882-3a091b0bce80 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:39.974724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2c2a7-639c-4891-8882-3a091b0bce80 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:39.975230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2c2a7-639c-4891-8882-3a091b0bce80 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:39.977032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 11/22] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:39.983176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df8344-62a0-4716-aa7f-ee8059735120 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:39.984014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df8344-62a0-4716-aa7f-ee8059735120 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:39.984496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df8344-62a0-4716-aa7f-ee8059735120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:40.010270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-48df8344-62a0-4716-aa7f-ee8059735120 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:40.011828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-48df8344-62a0-4716-aa7f-ee8059735120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:40.094006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 12/23] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:36:40.138741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2c2a7-639c-4891-8882-3a091b0bce80 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:40.286055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2c2a7-639c-4891-8882-3a091b0bce80 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:40.308582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:40.334839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a302b99-19ea-41bb-909b-ca54285d89fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:40.346073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a302b99-19ea-41bb-909b-ca54285d89fa None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:40.346073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a302b99-19ea-41bb-909b-ca54285d89fa None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:40.346073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a302b99-19ea-41bb-909b-ca54285d89fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:40.354146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a302b99-19ea-41bb-909b-ca54285d89fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-0a302b99-19ea-41bb-909b-ca54285d89fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:40.363926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:40.364709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:36:40.365889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:36:40.370734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 12/24] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:36:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:40.375863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df8344-62a0-4716-aa7f-ee8059735120 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:40.475111 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:40.507868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df8344-62a0-4716-aa7f-ee8059735120 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:40.563286 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f654d6-51f8-4ee9-bcc7-82088fc8c248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:40.564017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f654d6-51f8-4ee9-bcc7-82088fc8c248 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:40.564305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f654d6-51f8-4ee9-bcc7-82088fc8c248 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:40.564656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f654d6-51f8-4ee9-bcc7-82088fc8c248 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:40.565486 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f654d6-51f8-4ee9-bcc7-82088fc8c248 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:40.566410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f654d6-51f8-4ee9-bcc7-82088fc8c248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:40.567917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f654d6-51f8-4ee9-bcc7-82088fc8c248 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:40.581843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 13/25] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:36:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:40.587868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a302b99-19ea-41bb-909b-ca54285d89fa None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:40.649026 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec736f-e87e-4af6-9028-13cfa8af0bb3 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:40.649805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec736f-e87e-4af6-9028-13cfa8af0bb3 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:40.650477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec736f-e87e-4af6-9028-13cfa8af0bb3 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:40.652377 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 14/26] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:36:40.658277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5cfb4-5ff5-414e-a06f-91b94dff4faf None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:40.658843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5cfb4-5ff5-414e-a06f-91b94dff4faf None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:40.659563 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5cfb4-5ff5-414e-a06f-91b94dff4faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:40.686240 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-07d5cfb4-5ff5-414e-a06f-91b94dff4faf None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:40.687450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-07d5cfb4-5ff5-414e-a06f-91b94dff4faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:40.746165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a302b99-19ea-41bb-909b-ca54285d89fa None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:40.787560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e3743-a239-4f0b-baf7-566eadf229a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:40.788487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e3743-a239-4f0b-baf7-566eadf229a7 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:40.788787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e3743-a239-4f0b-baf7-566eadf229a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:40.789070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e3743-a239-4f0b-baf7-566eadf229a7 None admin] PATH_INFO: `/v3/users/88fee257b59d48f2b418ffa9e86b6f8e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:40.790985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e3743-a239-4f0b-baf7-566eadf229a7 None admin] RBAC: Authorizing `identity:get_user(user_id=88fee257b59d48f2b418ffa9e86b6f8e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:40.792513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e3743-a239-4f0b-baf7-566eadf229a7 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:40.798957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 13/27] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:40 2022] GET /identity/v3/users/88fee257b59d48f2b418ffa9e86b6f8e => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:40.800968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c6ec5-bb5a-4121-b9e3-0832f7c22139 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:40.801271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c6ec5-bb5a-4121-b9e3-0832f7c22139 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:40.801557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c6ec5-bb5a-4121-b9e3-0832f7c22139 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:40.803625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 14/28] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:40.805078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c6ec5-bb5a-4121-b9e3-0832f7c22139 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:40.847691 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 15/29] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:40.851647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d68be-e7ce-4b49-8a61-a8a58b8b1657 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:40.851945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d68be-e7ce-4b49-8a61-a8a58b8b1657 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:40.852425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d68be-e7ce-4b49-8a61-a8a58b8b1657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:40.880949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:40.890024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-461d68be-e7ce-4b49-8a61-a8a58b8b1657 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:40.891261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-461d68be-e7ce-4b49-8a61-a8a58b8b1657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:40.977660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c6ec5-bb5a-4121-b9e3-0832f7c22139 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:40.979031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:40.988552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 16/30] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:40.989602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d68be-e7ce-4b49-8a61-a8a58b8b1657 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:41.075370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11964ca-3f48-4afe-a78c-0030a2a597fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:41.075370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11964ca-3f48-4afe-a78c-0030a2a597fd None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:41.075370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11964ca-3f48-4afe-a78c-0030a2a597fd None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:41.075370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11964ca-3f48-4afe-a78c-0030a2a597fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:41.076866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11964ca-3f48-4afe-a78c-0030a2a597fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:41.078656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11964ca-3f48-4afe-a78c-0030a2a597fd None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:41.080616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 15/31] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:36:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:41.082161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11964ca-3f48-4afe-a78c-0030a2a597fd None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:41.110775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d68be-e7ce-4b49-8a61-a8a58b8b1657 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:41.149475 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e8e40a-1cc8-4b12-8690-28418085b2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:41.150278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8e40a-1cc8-4b12-8690-28418085b2a0 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:41.150600 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8e40a-1cc8-4b12-8690-28418085b2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:41.150832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e8e40a-1cc8-4b12-8690-28418085b2a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:41.161904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8e40a-1cc8-4b12-8690-28418085b2a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:41.163480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e8e40a-1cc8-4b12-8690-28418085b2a0 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-07e8e40a-1cc8-4b12-8690-28418085b2a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:36:41.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:41.184783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:41.190389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:36:41.190389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:36:41.190389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:36:41.190389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:36:41.190389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 17/32] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:36:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:41.190389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3c222-5613-4bd8-8e7f-6a5d3e848f1b None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:41.190389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3c222-5613-4bd8-8e7f-6a5d3e848f1b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:41.190389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3c222-5613-4bd8-8e7f-6a5d3e848f1b None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:41.190389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 18/33] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:41.191434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3c222-5613-4bd8-8e7f-6a5d3e848f1b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:41.202653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11964ca-3f48-4afe-a78c-0030a2a597fd None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:41.216411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:41.239481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41298fcd-2398-4f67-b753-72971f4f73dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:41.240409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41298fcd-2398-4f67-b753-72971f4f73dc None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:41.241231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41298fcd-2398-4f67-b753-72971f4f73dc None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:41.241746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41298fcd-2398-4f67-b753-72971f4f73dc None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:41.243002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41298fcd-2398-4f67-b753-72971f4f73dc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:41.244976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41298fcd-2398-4f67-b753-72971f4f73dc None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:41.248499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:41.301076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3c222-5613-4bd8-8e7f-6a5d3e848f1b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:41.339582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e22e7-8c3e-485b-84d4-d44d87ebbe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:41.341814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e22e7-8c3e-485b-84d4-d44d87ebbe0b None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:41.341814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e22e7-8c3e-485b-84d4-d44d87ebbe0b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:41.341814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e22e7-8c3e-485b-84d4-d44d87ebbe0b None admin] PATH_INFO: `/v3/domains/default/users/88fee257b59d48f2b418ffa9e86b6f8e/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:41.343346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e22e7-8c3e-485b-84d4-d44d87ebbe0b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=88fee257b59d48f2b418ffa9e86b6f8e, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:41.344935 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e22e7-8c3e-485b-84d4-d44d87ebbe0b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:41.444949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 16/34] 10.222.0.81 () {60 vars in 1131 bytes} [Thu Jun 16 18:36:41 2022] POST /identity/v3/roles => generated 219 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:41.445981 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41298fcd-2398-4f67-b753-72971f4f73dc None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:41.664036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 19/35] 10.222.0.81 () {58 vars in 1361 bytes} [Thu Jun 16 18:36:41 2022] PUT /identity/v3/domains/default/users/88fee257b59d48f2b418ffa9e86b6f8e/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:41.666325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57282f9-a38f-421a-99a1-0d71e596b0fd None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:41.666722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57282f9-a38f-421a-99a1-0d71e596b0fd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:41.667013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57282f9-a38f-421a-99a1-0d71e596b0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:41.672879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41298fcd-2398-4f67-b753-72971f4f73dc None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:41.701509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d57282f9-a38f-421a-99a1-0d71e596b0fd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:41.703097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d57282f9-a38f-421a-99a1-0d71e596b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:41.712889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514ff9d-ec64-453b-b0f1-2ef48f7270e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:41.713616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514ff9d-ec64-453b-b0f1-2ef48f7270e2 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:41.713910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514ff9d-ec64-453b-b0f1-2ef48f7270e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:41.714193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514ff9d-ec64-453b-b0f1-2ef48f7270e2 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:41.714977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514ff9d-ec64-453b-b0f1-2ef48f7270e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:41.715300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514ff9d-ec64-453b-b0f1-2ef48f7270e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:41.716924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514ff9d-ec64-453b-b0f1-2ef48f7270e2 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:41.730582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 17/36] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:41.740985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514ff9d-ec64-453b-b0f1-2ef48f7270e2 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:41.821335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:41.845783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 20/37] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:41.870462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57282f9-a38f-421a-99a1-0d71e596b0fd None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:41.937867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:41.964142 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514ff9d-ec64-453b-b0f1-2ef48f7270e2 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:42.007832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be66f69-4cb2-4b2d-9710-345eac89390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:42.008529 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be66f69-4cb2-4b2d-9710-345eac89390f None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:42.016834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be66f69-4cb2-4b2d-9710-345eac89390f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:42.016834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be66f69-4cb2-4b2d-9710-345eac89390f None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:42.017061 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57282f9-a38f-421a-99a1-0d71e596b0fd None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:42.017491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be66f69-4cb2-4b2d-9710-345eac89390f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:42.017491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be66f69-4cb2-4b2d-9710-345eac89390f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:42.045212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93598650-0f85-4fdd-86ae-8dab064ee47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:42.045918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93598650-0f85-4fdd-86ae-8dab064ee47d None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:42.046205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93598650-0f85-4fdd-86ae-8dab064ee47d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:42.046483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93598650-0f85-4fdd-86ae-8dab064ee47d None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:42.047237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93598650-0f85-4fdd-86ae-8dab064ee47d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:42.048979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93598650-0f85-4fdd-86ae-8dab064ee47d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:42.096452 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 18/38] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:36:41 2022] POST /identity/v3/projects => generated 301 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:42.097997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb16ab9-8743-4c0d-940d-05103389f54a None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:42.098298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb16ab9-8743-4c0d-940d-05103389f54a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:42.098582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb16ab9-8743-4c0d-940d-05103389f54a None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:42.100013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 19/39] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:42.101137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7737a-e5b2-42c9-9eec-fb95a76ba939 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:42.101509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7737a-e5b2-42c9-9eec-fb95a76ba939 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:42.101796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7737a-e5b2-42c9-9eec-fb95a76ba939 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:42.103130 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 20/40] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:42.105263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088cbb-ae62-424f-8861-a46bb7fcdf8b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:42.105836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088cbb-ae62-424f-8861-a46bb7fcdf8b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:42.106333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79088cbb-ae62-424f-8861-a46bb7fcdf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:42.127261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-79088cbb-ae62-424f-8861-a46bb7fcdf8b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:42.129874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-79088cbb-ae62-424f-8861-a46bb7fcdf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:42.130081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 21/41] 10.222.0.81 () {60 vars in 1131 bytes} [Thu Jun 16 18:36:41 2022] POST /identity/v3/roles => generated 225 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:36:42.131717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537563-c166-4427-ae41-a6a1b1ef7628 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:42.132063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537563-c166-4427-ae41-a6a1b1ef7628 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:42.132403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32537563-c166-4427-ae41-a6a1b1ef7628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:42.170701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-32537563-c166-4427-ae41-a6a1b1ef7628 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:42.172505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-32537563-c166-4427-ae41-a6a1b1ef7628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:42.303361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 21/42] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:36:42.318532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 22/43] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:42.328064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79088cbb-ae62-424f-8861-a46bb7fcdf8b None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:42.346019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:42.348535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32537563-c166-4427-ae41-a6a1b1ef7628 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:42.396603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:42.456996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79088cbb-ae62-424f-8861-a46bb7fcdf8b None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:42.480330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32537563-c166-4427-ae41-a6a1b1ef7628 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:42.499022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404d93d-b6d8-4773-bf39-e247e6b83c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:42.499708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404d93d-b6d8-4773-bf39-e247e6b83c56 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:42.500002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404d93d-b6d8-4773-bf39-e247e6b83c56 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:42.500289 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404d93d-b6d8-4773-bf39-e247e6b83c56 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:42.501047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404d93d-b6d8-4773-bf39-e247e6b83c56 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:42.503031 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404d93d-b6d8-4773-bf39-e247e6b83c56 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:42.516976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8516e5-6f77-4e7f-874f-b6ae0525720e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:42.517799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8516e5-6f77-4e7f-874f-b6ae0525720e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:42.518087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8516e5-6f77-4e7f-874f-b6ae0525720e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:42.519818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8516e5-6f77-4e7f-874f-b6ae0525720e None admin] PATH_INFO: `/v3/domains/default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:42.520269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8516e5-6f77-4e7f-874f-b6ae0525720e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:42.521759 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8516e5-6f77-4e7f-874f-b6ae0525720e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:42.523444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 23/44] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:36:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:42.529047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8516e5-6f77-4e7f-874f-b6ae0525720e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:42.553173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 22/45] 10.222.0.81 () {60 vars in 1131 bytes} [Thu Jun 16 18:36:42 2022] POST /identity/v3/roles => generated 223 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:42.637771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:42.644867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8516e5-6f77-4e7f-874f-b6ae0525720e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:42.680871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88290834-0834-42de-bbf0-5b0f7afba9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:42.681779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88290834-0834-42de-bbf0-5b0f7afba9f2 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:42.682069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88290834-0834-42de-bbf0-5b0f7afba9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:42.682343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88290834-0834-42de-bbf0-5b0f7afba9f2 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:42.683071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88290834-0834-42de-bbf0-5b0f7afba9f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:42.684564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88290834-0834-42de-bbf0-5b0f7afba9f2 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:42.788291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 24/46] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:36:42 2022] POST /identity/v3/projects => generated 312 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:36:44.014750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f319072-8ba8-4437-a1bc-c34440675a6a None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:44.015261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f319072-8ba8-4437-a1bc-c34440675a6a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:44.015342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f319072-8ba8-4437-a1bc-c34440675a6a None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:44.016710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 23/47] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:44.022533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1451f1-f65b-43fa-9f43-08702a7905bd None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:44.022951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1451f1-f65b-43fa-9f43-08702a7905bd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:44.023101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1451f1-f65b-43fa-9f43-08702a7905bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:44.039381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6d1451f1-f65b-43fa-9f43-08702a7905bd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:44.040561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6d1451f1-f65b-43fa-9f43-08702a7905bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:44.104187 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 25/48] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:44.127120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f319072-8ba8-4437-a1bc-c34440675a6a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:44.204164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:44.230204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f319072-8ba8-4437-a1bc-c34440675a6a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:44.263721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67cbaa-a61a-45a4-bcbd-3f65650a310b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:44.264513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67cbaa-a61a-45a4-bcbd-3f65650a310b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:44.264850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67cbaa-a61a-45a4-bcbd-3f65650a310b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:44.265080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67cbaa-a61a-45a4-bcbd-3f65650a310b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:44.275771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67cbaa-a61a-45a4-bcbd-3f65650a310b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-ff67cbaa-a61a-45a4-bcbd-3f65650a310b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:44.290913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:44.291775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:36:44.292517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:36:44.293093 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 24/49] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:36:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:44.299103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1451f1-f65b-43fa-9f43-08702a7905bd None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:44.376502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:44.397585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1451f1-f65b-43fa-9f43-08702a7905bd None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:44.431231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a556003-cdcb-4f46-b824-f076bc56888c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:44.431901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a556003-cdcb-4f46-b824-f076bc56888c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:44.432184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a556003-cdcb-4f46-b824-f076bc56888c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:44.432459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a556003-cdcb-4f46-b824-f076bc56888c None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:44.433191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a556003-cdcb-4f46-b824-f076bc56888c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:44.433527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a556003-cdcb-4f46-b824-f076bc56888c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:44.434919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a556003-cdcb-4f46-b824-f076bc56888c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:44.444353 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 26/50] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:36:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:44.449902 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67cbaa-a61a-45a4-bcbd-3f65650a310b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:44.548529 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67cbaa-a61a-45a4-bcbd-3f65650a310b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:44.582107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4098d-d434-4623-b8a6-2de546ffb183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:44.582982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4098d-d434-4623-b8a6-2de546ffb183 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:44.583274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4098d-d434-4623-b8a6-2de546ffb183 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:44.583557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4098d-d434-4623-b8a6-2de546ffb183 None admin] PATH_INFO: `/v3/users/88fee257b59d48f2b418ffa9e86b6f8e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:44.584747 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4098d-d434-4623-b8a6-2de546ffb183 None admin] RBAC: Authorizing `identity:get_user(user_id=88fee257b59d48f2b418ffa9e86b6f8e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:44.586261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4098d-d434-4623-b8a6-2de546ffb183 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:44.587883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 25/51] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:44 2022] GET /identity/v3/users/88fee257b59d48f2b418ffa9e86b6f8e => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:44.593478 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a556003-cdcb-4f46-b824-f076bc56888c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:44.690090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a556003-cdcb-4f46-b824-f076bc56888c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:44.721876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af8e50-d812-488d-8843-511d50c5e5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:44.723197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af8e50-d812-488d-8843-511d50c5e5d9 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:44.723487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af8e50-d812-488d-8843-511d50c5e5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:44.723765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af8e50-d812-488d-8843-511d50c5e5d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:44.724936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af8e50-d812-488d-8843-511d50c5e5d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:44.726434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af8e50-d812-488d-8843-511d50c5e5d9 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:44.728066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 27/52] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:36:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:44.734006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4098d-d434-4623-b8a6-2de546ffb183 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:44.736851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:44.833114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4098d-d434-4623-b8a6-2de546ffb183 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:44.868397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350eafcd-e0a2-4e9f-bc25-2f08bc46fea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:44.869062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350eafcd-e0a2-4e9f-bc25-2f08bc46fea8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:44.869386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350eafcd-e0a2-4e9f-bc25-2f08bc46fea8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:44.869668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350eafcd-e0a2-4e9f-bc25-2f08bc46fea8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:44.870512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350eafcd-e0a2-4e9f-bc25-2f08bc46fea8 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.domain.id=default, user.id=88fee257b59d48f2b418ffa9e86b6f8e) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:44.870832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350eafcd-e0a2-4e9f-bc25-2f08bc46fea8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:44.872279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350eafcd-e0a2-4e9f-bc25-2f08bc46fea8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:44.883328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 26/53] 10.222.0.81 () {56 vars in 1322 bytes} [Thu Jun 16 18:36:44 2022] GET /identity/v3/role_assignments?user.id=88fee257b59d48f2b418ffa9e86b6f8e&scope.domain.id=default&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:36:44.893397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8d6ae-2025-4179-a3f8-918edfd66076 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:44.893689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8d6ae-2025-4179-a3f8-918edfd66076 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:44.893969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8d6ae-2025-4179-a3f8-918edfd66076 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:44.895934 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 28/54] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:44.901787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146587-b535-4152-b092-cf97ca951ba0 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:44.902089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146587-b535-4152-b092-cf97ca951ba0 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:44.902371 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146587-b535-4152-b092-cf97ca951ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:44.920107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-8a146587-b535-4152-b092-cf97ca951ba0 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:44.921368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8a146587-b535-4152-b092-cf97ca951ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:44.984204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:44.992230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 27/55] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:45.015779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8d6ae-2025-4179-a3f8-918edfd66076 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:45.027481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:45.119004 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8d6ae-2025-4179-a3f8-918edfd66076 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:45.150462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5f0fd-0432-4b33-9675-9d3a600dc7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:45.151248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5f0fd-0432-4b33-9675-9d3a600dc7ea None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:45.151531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5f0fd-0432-4b33-9675-9d3a600dc7ea None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:45.151804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5f0fd-0432-4b33-9675-9d3a600dc7ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:45.152943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5f0fd-0432-4b33-9675-9d3a600dc7ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:45.154418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5f0fd-0432-4b33-9675-9d3a600dc7ea None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:45.156001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 29/56] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:36:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:45.161622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a146587-b535-4152-b092-cf97ca951ba0 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:45.256894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a146587-b535-4152-b092-cf97ca951ba0 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:45.289206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4482c-e81d-4c10-a683-312fc3b17914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:45.289881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4482c-e81d-4c10-a683-312fc3b17914 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:45.290165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4482c-e81d-4c10-a683-312fc3b17914 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:45.290438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4482c-e81d-4c10-a683-312fc3b17914 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:45.291136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4482c-e81d-4c10-a683-312fc3b17914 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:45.292575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa4482c-e81d-4c10-a683-312fc3b17914 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:45.387836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 28/57] 10.222.0.81 () {60 vars in 1140 bytes} [Thu Jun 16 18:36:45 2022] POST /identity/v3/projects => generated 298 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:47.478437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617105b-b6d6-42c2-aec0-f5d82703f44d None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:47.478939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617105b-b6d6-42c2-aec0-f5d82703f44d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:47.479026 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617105b-b6d6-42c2-aec0-f5d82703f44d None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:47.480379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 30/58] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:47.485641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9409420-9e61-4e0a-a5be-0dc06c4645d1 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:47.486039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9409420-9e61-4e0a-a5be-0dc06c4645d1 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:47.486210 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9409420-9e61-4e0a-a5be-0dc06c4645d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:47.502415 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d9409420-9e61-4e0a-a5be-0dc06c4645d1 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:47.503599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d9409420-9e61-4e0a-a5be-0dc06c4645d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:47.545869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:47.571428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 29/59] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:47.597120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617105b-b6d6-42c2-aec0-f5d82703f44d None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:47.672731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:47.695107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617105b-b6d6-42c2-aec0-f5d82703f44d None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:47.726807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337286a-50a1-451a-9f16-030520a877b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:47.727592 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337286a-50a1-451a-9f16-030520a877b1 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:47.727881 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337286a-50a1-451a-9f16-030520a877b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:47.728162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337286a-50a1-451a-9f16-030520a877b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:47.729333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337286a-50a1-451a-9f16-030520a877b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:47.730801 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337286a-50a1-451a-9f16-030520a877b1 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:47.732398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 31/60] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:36:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:47.738161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9409420-9e61-4e0a-a5be-0dc06c4645d1 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:47.838232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9409420-9e61-4e0a-a5be-0dc06c4645d1 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:47.882784 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00e880-3466-42cd-bb9e-8e0f74e4405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:47.883482 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00e880-3466-42cd-bb9e-8e0f74e4405f None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:47.883768 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00e880-3466-42cd-bb9e-8e0f74e4405f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:47.884042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00e880-3466-42cd-bb9e-8e0f74e4405f None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:47.884749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00e880-3466-42cd-bb9e-8e0f74e4405f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:47.886234 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00e880-3466-42cd-bb9e-8e0f74e4405f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:47.975656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 30/61] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:36:47 2022] POST /identity/v3/users => generated 275 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:36:51.397119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400ab4d-d033-47d2-bce2-1858de1a191a None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:51.397119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400ab4d-d033-47d2-bce2-1858de1a191a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:51.397119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400ab4d-d033-47d2-bce2-1858de1a191a None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:51.397119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 32/62] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:51.403102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced518eb-4589-485b-9418-3e6abd2b0eeb None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:51.403598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced518eb-4589-485b-9418-3e6abd2b0eeb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:51.403804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced518eb-4589-485b-9418-3e6abd2b0eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:51.426348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ced518eb-4589-485b-9418-3e6abd2b0eeb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:51.427602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ced518eb-4589-485b-9418-3e6abd2b0eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:51.462125 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0babc80-4e78-49ca-a375-59653ec33abf None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:51.462754 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0babc80-4e78-49ca-a375-59653ec33abf None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:51.463252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0babc80-4e78-49ca-a375-59653ec33abf None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:51.464841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 33/63] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:51.469274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:51.477161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b83d2c-a32b-491c-81dd-f775a0cc3dff None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:51.477499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b83d2c-a32b-491c-81dd-f775a0cc3dff None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:51.477987 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b83d2c-a32b-491c-81dd-f775a0cc3dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:51.502233 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-c1b83d2c-a32b-491c-81dd-f775a0cc3dff None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:51.505225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c1b83d2c-a32b-491c-81dd-f775a0cc3dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:51.538859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 31/64] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:51.571940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced518eb-4589-485b-9418-3e6abd2b0eeb None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:51.592029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 34/65] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:51.602070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086bc1c-42c1-49fc-a095-cf9030668f10 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:51.602581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086bc1c-42c1-49fc-a095-cf9030668f10 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:51.603003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086bc1c-42c1-49fc-a095-cf9030668f10 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:51.604536 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 35/66] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:36:51.615705 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c63c0-8aac-47fe-8ca1-ed88920e25a2 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:51.616206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c63c0-8aac-47fe-8ca1-ed88920e25a2 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:51.621732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c63c0-8aac-47fe-8ca1-ed88920e25a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:51.640794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6c8c63c0-8aac-47fe-8ca1-ed88920e25a2 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:51.642153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6c8c63c0-8aac-47fe-8ca1-ed88920e25a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:51.702496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced518eb-4589-485b-9418-3e6abd2b0eeb None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:51.715988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 36/67] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:36:51.726180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c63c0-8aac-47fe-8ca1-ed88920e25a2 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:51.730333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:51.742159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24608c8-5ad6-4b15-a797-e39bdc6bf29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:51.743371 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24608c8-5ad6-4b15-a797-e39bdc6bf29a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:51.744203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24608c8-5ad6-4b15-a797-e39bdc6bf29a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:51.747096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24608c8-5ad6-4b15-a797-e39bdc6bf29a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:51.759207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24608c8-5ad6-4b15-a797-e39bdc6bf29a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-d24608c8-5ad6-4b15-a797-e39bdc6bf29a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:51.768631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:51.769564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:36:51.770384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 32/68] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:36:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:51.771966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f85ae-5ab2-43e5-928f-b81c576747f2 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:51.771966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f85ae-5ab2-43e5-928f-b81c576747f2 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:51.771966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f85ae-5ab2-43e5-928f-b81c576747f2 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:51.773282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 33/69] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:51.778289 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f85ae-5ab2-43e5-928f-b81c576747f2 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:51.802706 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:51.838614 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8c63c0-8aac-47fe-8ca1-ed88920e25a2 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:51.880731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0d14b-bf30-4864-9db7-4acf29b7f2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:51.881520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0d14b-bf30-4864-9db7-4acf29b7f2e7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:51.881806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0d14b-bf30-4864-9db7-4acf29b7f2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:51.882093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0d14b-bf30-4864-9db7-4acf29b7f2e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:51.887957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f85ae-5ab2-43e5-928f-b81c576747f2 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:51.894049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0d14b-bf30-4864-9db7-4acf29b7f2e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-c2f0d14b-bf30-4864-9db7-4acf29b7f2e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:51.901832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:51.902671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:36:51.903484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 37/70] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:36:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:51.904091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0d14b-bf30-4864-9db7-4acf29b7f2e7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:51.933842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd9f92-774c-459e-9cda-abd5c67caf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:51.934691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9f92-774c-459e-9cda-abd5c67caf36 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:51.934975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9f92-774c-459e-9cda-abd5c67caf36 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:51.935255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9f92-774c-459e-9cda-abd5c67caf36 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:51.946868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd9f92-774c-459e-9cda-abd5c67caf36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-f7dd9f92-774c-459e-9cda-abd5c67caf36 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:51.954856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:51.955883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:36:51.956652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 34/71] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:36:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:51.957342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f8a1-2422-4834-8dc8-333b69a8a4c1 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:51.957534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f8a1-2422-4834-8dc8-333b69a8a4c1 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:51.957834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2328f8a1-2422-4834-8dc8-333b69a8a4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:51.972209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:51.974297 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-2328f8a1-2422-4834-8dc8-333b69a8a4c1 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:51.975503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-2328f8a1-2422-4834-8dc8-333b69a8a4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:52.011691 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0d14b-bf30-4864-9db7-4acf29b7f2e7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:52.045172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5a0ee-d934-4aa0-8ddb-f8544c383bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:52.050936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 35/72] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:52.051086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5a0ee-d934-4aa0-8ddb-f8544c383bb3 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.051086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5a0ee-d934-4aa0-8ddb-f8544c383bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.051086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5a0ee-d934-4aa0-8ddb-f8544c383bb3 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.051858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e65bca-06b6-44e1-a5f7-7ecbbdea07be None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.051858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e65bca-06b6-44e1-a5f7-7ecbbdea07be None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.051858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e65bca-06b6-44e1-a5f7-7ecbbdea07be None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.052027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5a0ee-d934-4aa0-8ddb-f8544c383bb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:52.052027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5a0ee-d934-4aa0-8ddb-f8544c383bb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:52.052151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 36/73] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:52.052214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5a0ee-d934-4aa0-8ddb-f8544c383bb3 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:52.052276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e65bca-06b6-44e1-a5f7-7ecbbdea07be None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:52.063801 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 38/74] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:36:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:52.063801 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5a0ee-d934-4aa0-8ddb-f8544c383bb3 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:52.071475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:52.156344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e65bca-06b6-44e1-a5f7-7ecbbdea07be None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:52.159307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5a0ee-d934-4aa0-8ddb-f8544c383bb3 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:52.184293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:52.191028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22acbb4-4c8c-4774-ad22-03a67e410f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:52.191786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22acbb4-4c8c-4774-ad22-03a67e410f45 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.191986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22acbb4-4c8c-4774-ad22-03a67e410f45 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.192268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22acbb4-4c8c-4774-ad22-03a67e410f45 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.192532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5462ba7-79dd-4a8b-a524-f311625ee8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:52.193035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22acbb4-4c8c-4774-ad22-03a67e410f45 None admin] RBAC: Adding query filter params (name=member) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:52.193199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462ba7-79dd-4a8b-a524-f311625ee8d7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.193377 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22acbb4-4c8c-4774-ad22-03a67e410f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:52.193518 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462ba7-79dd-4a8b-a524-f311625ee8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.193800 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5462ba7-79dd-4a8b-a524-f311625ee8d7 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.194561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5462ba7-79dd-4a8b-a524-f311625ee8d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:52.194773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22acbb4-4c8c-4774-ad22-03a67e410f45 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:52.194860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5462ba7-79dd-4a8b-a524-f311625ee8d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:52.196232 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5462ba7-79dd-4a8b-a524-f311625ee8d7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:52.204349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 39/75] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:36:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:52.206205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 37/76] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:36:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:52.210700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5adbfb-af0d-4716-8774-062c3f8f39da None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.210700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5adbfb-af0d-4716-8774-062c3f8f39da None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.210892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22acbb4-4c8c-4774-ad22-03a67e410f45 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:52.210960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5adbfb-af0d-4716-8774-062c3f8f39da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.225685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-0f5adbfb-af0d-4716-8774-062c3f8f39da None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:52.227307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0f5adbfb-af0d-4716-8774-062c3f8f39da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:52.291083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:52.293729 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 40/77] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:52.294585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5adbfb-af0d-4716-8774-062c3f8f39da None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:52.320784 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22acbb4-4c8c-4774-ad22-03a67e410f45 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:52.366195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62b3d2-ede5-4006-b1b5-65e84125aaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:52.367085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62b3d2-ede5-4006-b1b5-65e84125aaa9 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.367370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62b3d2-ede5-4006-b1b5-65e84125aaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.367649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62b3d2-ede5-4006-b1b5-65e84125aaa9 None admin] PATH_INFO: `/v3/users/88fee257b59d48f2b418ffa9e86b6f8e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.368870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62b3d2-ede5-4006-b1b5-65e84125aaa9 None admin] RBAC: Authorizing `identity:get_user(user_id=88fee257b59d48f2b418ffa9e86b6f8e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:52.370355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62b3d2-ede5-4006-b1b5-65e84125aaa9 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:52.372000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 38/78] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:52 2022] GET /identity/v3/users/88fee257b59d48f2b418ffa9e86b6f8e => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:52.372757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62b3d2-ede5-4006-b1b5-65e84125aaa9 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:52.401968 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5adbfb-af0d-4716-8774-062c3f8f39da None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:52.419095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:52.435338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9dca6-b902-4c75-9962-2124b61eb0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:52.436130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9dca6-b902-4c75-9962-2124b61eb0a3 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.436420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9dca6-b902-4c75-9962-2124b61eb0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.436707 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9dca6-b902-4c75-9962-2124b61eb0a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.447841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9dca6-b902-4c75-9962-2124b61eb0a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-0fe9dca6-b902-4c75-9962-2124b61eb0a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:52.455821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:52.456662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:36:52.457998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 41/79] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:36:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:52.458803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9dca6-b902-4c75-9962-2124b61eb0a3 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:52.473953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62b3d2-ede5-4006-b1b5-65e84125aaa9 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:52.506913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2f4ec-d9e3-40be-87ea-9e9c0e204e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:52.507784 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2f4ec-d9e3-40be-87ea-9e9c0e204e37 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.508076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2f4ec-d9e3-40be-87ea-9e9c0e204e37 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.508372 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2f4ec-d9e3-40be-87ea-9e9c0e204e37 None admin] PATH_INFO: `/v3/users/cf1ee67d28b6436fa3ec26eeae38390a` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.523310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2f4ec-d9e3-40be-87ea-9e9c0e204e37 None admin] RBAC: Authorizing `identity:get_user(user_id=cf1ee67d28b6436fa3ec26eeae38390a)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:52.524785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2f4ec-d9e3-40be-87ea-9e9c0e204e37 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:52.526472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 39/80] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:52 2022] GET /identity/v3/users/cf1ee67d28b6436fa3ec26eeae38390a => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:52.527238 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2f4ec-d9e3-40be-87ea-9e9c0e204e37 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:52.529968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:52.556407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9dca6-b902-4c75-9962-2124b61eb0a3 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:52.589403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232b7e9-d7ac-4830-8b75-4a90a1e26495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:52.590276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232b7e9-d7ac-4830-8b75-4a90a1e26495 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.590564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232b7e9-d7ac-4830-8b75-4a90a1e26495 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.590849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232b7e9-d7ac-4830-8b75-4a90a1e26495 None admin] PATH_INFO: `/v3/users/cf1ee67d28b6436fa3ec26eeae38390a` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.592031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232b7e9-d7ac-4830-8b75-4a90a1e26495 None admin] RBAC: Authorizing `identity:get_user(user_id=cf1ee67d28b6436fa3ec26eeae38390a)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:52.593564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232b7e9-d7ac-4830-8b75-4a90a1e26495 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:52.595192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 42/81] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:52 2022] GET /identity/v3/users/cf1ee67d28b6436fa3ec26eeae38390a => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:52.600473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232b7e9-d7ac-4830-8b75-4a90a1e26495 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:52.627673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2f4ec-d9e3-40be-87ea-9e9c0e204e37 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:52.656042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:52.665433 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6aa312-e4c8-4319-a184-e1233931568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:52.668229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa312-e4c8-4319-a184-e1233931568d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.668229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa312-e4c8-4319-a184-e1233931568d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.668229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa312-e4c8-4319-a184-e1233931568d None admin] PATH_INFO: `/v3/domains/default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.668229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6aa312-e4c8-4319-a184-e1233931568d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:52.669439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6aa312-e4c8-4319-a184-e1233931568d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:52.670990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 40/82] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:36:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:52.674089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6aa312-e4c8-4319-a184-e1233931568d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:52.709050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232b7e9-d7ac-4830-8b75-4a90a1e26495 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:52.742204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87408126-c7ba-4960-b4fa-1dd86bad2a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:52.742860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87408126-c7ba-4960-b4fa-1dd86bad2a0e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.743150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87408126-c7ba-4960-b4fa-1dd86bad2a0e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.743431 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87408126-c7ba-4960-b4fa-1dd86bad2a0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.744023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:52.774323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6aa312-e4c8-4319-a184-e1233931568d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:52.807002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdeba6-59fc-4db2-b223-b047d89a3320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:52.807665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdeba6-59fc-4db2-b223-b047d89a3320 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.807950 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdeba6-59fc-4db2-b223-b047d89a3320 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.808229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bdeba6-59fc-4db2-b223-b047d89a3320 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:52.808843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87408126-c7ba-4960-b4fa-1dd86bad2a0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:52.808962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdeba6-59fc-4db2-b223-b047d89a3320 None admin] RBAC: Adding query filter params (name=member) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:52.809373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdeba6-59fc-4db2-b223-b047d89a3320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:52.810384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87408126-c7ba-4960-b4fa-1dd86bad2a0e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:52.810729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bdeba6-59fc-4db2-b223-b047d89a3320 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:52.819204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 41/83] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:36:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:52.820141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdeba6-59fc-4db2-b223-b047d89a3320 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:52.854907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:52.909875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 43/84] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:36:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:36:52.910721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87408126-c7ba-4960-b4fa-1dd86bad2a0e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:52.921000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bdeba6-59fc-4db2-b223-b047d89a3320 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:52.950045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:52.955599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3bc74-8f35-4df4-b9f4-7b9887140b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:52.956251 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3bc74-8f35-4df4-b9f4-7b9887140b32 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:52.956545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3bc74-8f35-4df4-b9f4-7b9887140b32 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:52.956828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3bc74-8f35-4df4-b9f4-7b9887140b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:53.006309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87408126-c7ba-4960-b4fa-1dd86bad2a0e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:53.023058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3bc74-8f35-4df4-b9f4-7b9887140b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:53.024524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3bc74-8f35-4df4-b9f4-7b9887140b32 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:53.039649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a10a-de36-476e-b1ce-bf2f8843a3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:53.040294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a10a-de36-476e-b1ce-bf2f8843a3ee None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:53.040580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a10a-de36-476e-b1ce-bf2f8843a3ee None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:53.040861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a10a-de36-476e-b1ce-bf2f8843a3ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:53.055379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:53.108110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080a10a-de36-476e-b1ce-bf2f8843a3ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:53.109847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080a10a-de36-476e-b1ce-bf2f8843a3ee None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:53.119676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 42/85] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:36:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:36:53.120545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3bc74-8f35-4df4-b9f4-7b9887140b32 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:53.150667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:53.207391 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 44/86] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:36:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:36:53.208558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a10a-de36-476e-b1ce-bf2f8843a3ee None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:53.220570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3bc74-8f35-4df4-b9f4-7b9887140b32 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:53.253826 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19ba81-7368-40d6-b614-0d00f915e121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:53.254057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:53.255060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19ba81-7368-40d6-b614-0d00f915e121 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:53.255355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19ba81-7368-40d6-b614-0d00f915e121 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:53.255627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19ba81-7368-40d6-b614-0d00f915e121 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:53.256330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19ba81-7368-40d6-b614-0d00f915e121 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:53.257884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19ba81-7368-40d6-b614-0d00f915e121 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:53.305367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a10a-de36-476e-b1ce-bf2f8843a3ee None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:53.329073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 43/87] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:36:53 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:53.335396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19ba81-7368-40d6-b614-0d00f915e121 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:53.345937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa41e20-af69-473c-8cbc-0f40fe93045d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:53.346845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa41e20-af69-473c-8cbc-0f40fe93045d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:53.347134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa41e20-af69-473c-8cbc-0f40fe93045d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:53.347415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa41e20-af69-473c-8cbc-0f40fe93045d None admin] PATH_INFO: `/v3/users/cf1ee67d28b6436fa3ec26eeae38390a` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:53.348808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa41e20-af69-473c-8cbc-0f40fe93045d None admin] RBAC: Authorizing `identity:get_user(user_id=cf1ee67d28b6436fa3ec26eeae38390a)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:53.350412 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa41e20-af69-473c-8cbc-0f40fe93045d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:53.352194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 45/88] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:53 2022] GET /identity/v3/users/cf1ee67d28b6436fa3ec26eeae38390a => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:53.353078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa41e20-af69-473c-8cbc-0f40fe93045d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:53.456939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:53.470208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19ba81-7368-40d6-b614-0d00f915e121 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:53.470860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa41e20-af69-473c-8cbc-0f40fe93045d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:53.486032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:53.512011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a0fcc-4f3f-42ea-afdc-e122e3a22260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:53.512944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a0fcc-4f3f-42ea-afdc-e122e3a22260 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:53.513255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a0fcc-4f3f-42ea-afdc-e122e3a22260 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:53.513416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa1db4-3476-4d62-8edf-b922426abd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:53.513854 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078a0fcc-4f3f-42ea-afdc-e122e3a22260 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:53.514298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa1db4-3476-4d62-8edf-b922426abd9c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:53.515222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa1db4-3476-4d62-8edf-b922426abd9c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:53.515320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a0fcc-4f3f-42ea-afdc-e122e3a22260 None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:53.515403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfa1db4-3476-4d62-8edf-b922426abd9c None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:53.516926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078a0fcc-4f3f-42ea-afdc-e122e3a22260 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:53.517029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa1db4-3476-4d62-8edf-b922426abd9c None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:53.518064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfa1db4-3476-4d62-8edf-b922426abd9c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:53.519780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 44/89] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:36:53 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:53.520593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa1db4-3476-4d62-8edf-b922426abd9c None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:53.523526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 46/90] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:36:53 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:53.529637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a0fcc-4f3f-42ea-afdc-e122e3a22260 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:53.641825 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfa1db4-3476-4d62-8edf-b922426abd9c None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:53.653079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078a0fcc-4f3f-42ea-afdc-e122e3a22260 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:53.687194 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbac58-bca0-4542-815f-888f6a2fa96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:53.688475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbac58-bca0-4542-815f-888f6a2fa96a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:53.688768 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbac58-bca0-4542-815f-888f6a2fa96a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:53.689054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbac58-bca0-4542-815f-888f6a2fa96a None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:53.690660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbac58-bca0-4542-815f-888f6a2fa96a None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:53.692394 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbac58-bca0-4542-815f-888f6a2fa96a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:53.694102 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719813d-2b2b-435c-81ec-980a07a7173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:53.694260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 45/91] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:36:53 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:53.695016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbac58-bca0-4542-815f-888f6a2fa96a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:53.695282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719813d-2b2b-435c-81ec-980a07a7173f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:53.698957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719813d-2b2b-435c-81ec-980a07a7173f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:53.698957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719813d-2b2b-435c-81ec-980a07a7173f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:53.748351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:53.773721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:53.794459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719813d-2b2b-435c-81ec-980a07a7173f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:53.796446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719813d-2b2b-435c-81ec-980a07a7173f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:53.826660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbac58-bca0-4542-815f-888f6a2fa96a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:53.867163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2228e01-1011-4512-a719-72c5a259df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:53.868412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2228e01-1011-4512-a719-72c5a259df46 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:53.868715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2228e01-1011-4512-a719-72c5a259df46 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:53.868997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2228e01-1011-4512-a719-72c5a259df46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:53.869865 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2228e01-1011-4512-a719-72c5a259df46 None admin] RBAC: Adding query filter params (role.id=0960434fd9da404ea96841eb4306eaae, scope.project.id=0e628770474b423e99809dd568e34a0e, user.id=cf1ee67d28b6436fa3ec26eeae38390a) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:53.870187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2228e01-1011-4512-a719-72c5a259df46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:53.872162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2228e01-1011-4512-a719-72c5a259df46 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:53.882368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 46/92] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:36:53 2022] GET /identity/v3/role_assignments?user.id=cf1ee67d28b6436fa3ec26eeae38390a&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=0960434fd9da404ea96841eb4306eaae => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:53.889519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2228e01-1011-4512-a719-72c5a259df46 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:53.942338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 47/93] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:36:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:36:53.951249 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719813d-2b2b-435c-81ec-980a07a7173f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:53.993733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:54.008716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2228e01-1011-4512-a719-72c5a259df46 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:54.048700 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9cbaf-46da-4e21-86dc-6afa672be3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:54.049427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9cbaf-46da-4e21-86dc-6afa672be3ee None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:54.049872 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9cbaf-46da-4e21-86dc-6afa672be3ee None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:54.050164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9cbaf-46da-4e21-86dc-6afa672be3ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:54.051014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9cbaf-46da-4e21-86dc-6afa672be3ee None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=0e628770474b423e99809dd568e34a0e, user.id=88fee257b59d48f2b418ffa9e86b6f8e) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:54.051379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9cbaf-46da-4e21-86dc-6afa672be3ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:54.052868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9cbaf-46da-4e21-86dc-6afa672be3ee None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:54.060070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 47/94] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:36:53 2022] GET /identity/v3/role_assignments?user.id=88fee257b59d48f2b418ffa9e86b6f8e&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:54.061114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9cbaf-46da-4e21-86dc-6afa672be3ee None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:54.178708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9cbaf-46da-4e21-86dc-6afa672be3ee None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:54.205168 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:54.218379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a674fb-aadf-4a77-a88b-91b680205fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:54.219307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a674fb-aadf-4a77-a88b-91b680205fba None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:54.219600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a674fb-aadf-4a77-a88b-91b680205fba None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:54.219910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a674fb-aadf-4a77-a88b-91b680205fba None admin] PATH_INFO: `/v3/projects/78876b2fbff64f139ae1172647ebbc4e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:54.221124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a674fb-aadf-4a77-a88b-91b680205fba None admin] RBAC: Authorizing `identity:get_project(project_id=78876b2fbff64f139ae1172647ebbc4e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:54.223889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a674fb-aadf-4a77-a88b-91b680205fba None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:54.225098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 48/95] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:36:54 2022] GET /identity/v3/projects/78876b2fbff64f139ae1172647ebbc4e => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:54.231284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a674fb-aadf-4a77-a88b-91b680205fba None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:54.232880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719813d-2b2b-435c-81ec-980a07a7173f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:54.265741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:54.302027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50803692-39ba-411d-aff9-a551e7da9670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:54.303591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50803692-39ba-411d-aff9-a551e7da9670 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:54.304048 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50803692-39ba-411d-aff9-a551e7da9670 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:54.304649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50803692-39ba-411d-aff9-a551e7da9670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:54.310845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50803692-39ba-411d-aff9-a551e7da9670 None admin] RBAC: Adding query filter params (role.id=5f24daf2ba2b4f8fab95c87cefb52231, scope.project.id=0e628770474b423e99809dd568e34a0e, user.id=cf1ee67d28b6436fa3ec26eeae38390a) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:54.310845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50803692-39ba-411d-aff9-a551e7da9670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:54.310845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50803692-39ba-411d-aff9-a551e7da9670 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:54.316328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 48/96] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:36:53 2022] GET /identity/v3/role_assignments?user.id=cf1ee67d28b6436fa3ec26eeae38390a&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=5f24daf2ba2b4f8fab95c87cefb52231 => generated 259 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:54.351219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a674fb-aadf-4a77-a88b-91b680205fba None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:54.391088 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e51f5-11cf-499c-8110-b28d2e0a8227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:54.391762 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e51f5-11cf-499c-8110-b28d2e0a8227 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:54.392050 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e51f5-11cf-499c-8110-b28d2e0a8227 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:54.392331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e51f5-11cf-499c-8110-b28d2e0a8227 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:54.393172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e51f5-11cf-499c-8110-b28d2e0a8227 None admin] RBAC: Adding query filter params (role.id=0960434fd9da404ea96841eb4306eaae, scope.project.id=78876b2fbff64f139ae1172647ebbc4e, user.id=cf1ee67d28b6436fa3ec26eeae38390a) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:54.393565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e51f5-11cf-499c-8110-b28d2e0a8227 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:54.395149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397e51f5-11cf-499c-8110-b28d2e0a8227 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:54.402082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 49/97] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:36:54 2022] GET /identity/v3/role_assignments?user.id=cf1ee67d28b6436fa3ec26eeae38390a&scope.project.id=78876b2fbff64f139ae1172647ebbc4e&role.id=0960434fd9da404ea96841eb4306eaae => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:36:56.043477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b6161-564a-41b2-82da-35df1f52990c None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.043477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b6161-564a-41b2-82da-35df1f52990c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.043477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b6161-564a-41b2-82da-35df1f52990c None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.043477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 49/98] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:56.050572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa908e43-d247-45a8-93ba-d02fd5f40eb7 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.050572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa908e43-d247-45a8-93ba-d02fd5f40eb7 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.050572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa908e43-d247-45a8-93ba-d02fd5f40eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.074183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-aa908e43-d247-45a8-93ba-d02fd5f40eb7 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:56.075803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-aa908e43-d247-45a8-93ba-d02fd5f40eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:56.195780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 50/99] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:56.230393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b6161-564a-41b2-82da-35df1f52990c None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:56.338860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c359408-d050-400f-8c4f-0273dfe786bf None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.339172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c359408-d050-400f-8c4f-0273dfe786bf None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.341028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c359408-d050-400f-8c4f-0273dfe786bf None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.341028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 51/100] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:56.354628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36e0e1-f931-43a5-b37f-a1423fea40ed None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.354919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36e0e1-f931-43a5-b37f-a1423fea40ed None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.355203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36e0e1-f931-43a5-b37f-a1423fea40ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.413486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-cf36e0e1-f931-43a5-b37f-a1423fea40ed None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:56.414730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-cf36e0e1-f931-43a5-b37f-a1423fea40ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:56.435110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b6161-564a-41b2-82da-35df1f52990c None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:56.469625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:56.546674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c04424-d8a0-4d9e-8226-5092aca1e528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:56.548078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c04424-d8a0-4d9e-8226-5092aca1e528 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.548369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c04424-d8a0-4d9e-8226-5092aca1e528 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.548648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c04424-d8a0-4d9e-8226-5092aca1e528 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.550202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c04424-d8a0-4d9e-8226-5092aca1e528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:56.555930 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c04424-d8a0-4d9e-8226-5092aca1e528 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:56.558262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 50/101] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:56.563840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c04424-d8a0-4d9e-8226-5092aca1e528 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:56.589872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:56.606042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 52/102] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:56.697881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36e0e1-f931-43a5-b37f-a1423fea40ed None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:56.705268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c04424-d8a0-4d9e-8226-5092aca1e528 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:56.840734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d8103-e327-4b34-a17f-ecbd2d8f35b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:56.841346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d8103-e327-4b34-a17f-ecbd2d8f35b4 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.841814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d8103-e327-4b34-a17f-ecbd2d8f35b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.842221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d8103-e327-4b34-a17f-ecbd2d8f35b4 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.843050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d8103-e327-4b34-a17f-ecbd2d8f35b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:56.844638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d8103-e327-4b34-a17f-ecbd2d8f35b4 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:56.927116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36e0e1-f931-43a5-b37f-a1423fea40ed None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:56.961495 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 51/103] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:36:56 2022] POST /identity/v3/projects => generated 302 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:36:56.965388 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e28e3-54b0-4233-8c45-7de4c38e64a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:56.966347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e28e3-54b0-4233-8c45-7de4c38e64a7 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:56.966827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e28e3-54b0-4233-8c45-7de4c38e64a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:56.967224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e28e3-54b0-4233-8c45-7de4c38e64a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:56.980038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e28e3-54b0-4233-8c45-7de4c38e64a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-6e6e28e3-54b0-4233-8c45-7de4c38e64a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:56.988424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:56.989849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:36:56.990638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 53/104] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:36:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:56.996689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d8103-e327-4b34-a17f-ecbd2d8f35b4 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.084954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:57.128613 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d8103-e327-4b34-a17f-ecbd2d8f35b4 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:57.179150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d124a0f-b507-460c-94dc-433a3f2c634c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:57.179888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d124a0f-b507-460c-94dc-433a3f2c634c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.180179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d124a0f-b507-460c-94dc-433a3f2c634c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.180457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d124a0f-b507-460c-94dc-433a3f2c634c None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.181332 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d124a0f-b507-460c-94dc-433a3f2c634c None admin] RBAC: Adding query filter params (name=reader) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:57.181650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d124a0f-b507-460c-94dc-433a3f2c634c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:57.183067 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d124a0f-b507-460c-94dc-433a3f2c634c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:57.201815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 52/105] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:36:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:57.207617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e28e3-54b0-4233-8c45-7de4c38e64a7 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.212238 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:57.320640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e28e3-54b0-4233-8c45-7de4c38e64a7 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:57.340631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0d06a-ff24-44ba-b7e3-1002a3135a99 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.340935 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0d06a-ff24-44ba-b7e3-1002a3135a99 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.341276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0d06a-ff24-44ba-b7e3-1002a3135a99 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.345281 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 53/106] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:36:57.351255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40066a-4bf6-44c5-868b-c397c1383315 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.351255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40066a-4bf6-44c5-868b-c397c1383315 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.351255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40066a-4bf6-44c5-868b-c397c1383315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.378324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ce40066a-4bf6-44c5-868b-c397c1383315 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:57.380487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ce40066a-4bf6-44c5-868b-c397c1383315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:57.389276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a849a4-9465-49b5-8276-4b9de2e52ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:57.390177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a849a4-9465-49b5-8276-4b9de2e52ddb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.390464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a849a4-9465-49b5-8276-4b9de2e52ddb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.390742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a849a4-9465-49b5-8276-4b9de2e52ddb None admin] PATH_INFO: `/v3/users/3628f92de2bf465d90beaeb1f935f0c7` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.416106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a849a4-9465-49b5-8276-4b9de2e52ddb None admin] RBAC: Authorizing `identity:get_user(user_id=3628f92de2bf465d90beaeb1f935f0c7)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:57.418384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a849a4-9465-49b5-8276-4b9de2e52ddb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:57.420290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 54/107] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:57 2022] GET /identity/v3/users/3628f92de2bf465d90beaeb1f935f0c7 => generated 289 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:57.426221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a849a4-9465-49b5-8276-4b9de2e52ddb None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.478306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 54/108] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:57.482584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b279a8-b6a8-4815-abc4-1d5d8c83354b None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.482584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b279a8-b6a8-4815-abc4-1d5d8c83354b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.482584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b279a8-b6a8-4815-abc4-1d5d8c83354b None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.482584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 55/109] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:57.489618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b15df-52e9-458d-8c80-bb8442fdf5bd None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.489902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b15df-52e9-458d-8c80-bb8442fdf5bd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.490188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b15df-52e9-458d-8c80-bb8442fdf5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.516604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-451b15df-52e9-458d-8c80-bb8442fdf5bd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:57.518247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-451b15df-52e9-458d-8c80-bb8442fdf5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:57.568640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:57.589903 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:57.607647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 56/110] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:36:57.609620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b15df-52e9-458d-8c80-bb8442fdf5bd None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.665000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a849a4-9465-49b5-8276-4b9de2e52ddb None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:57.733963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bac903-22ba-4d7e-bba0-e2446fc96725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:57.734617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bac903-22ba-4d7e-bba0-e2446fc96725 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.735163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bac903-22ba-4d7e-bba0-e2446fc96725 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.735449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bac903-22ba-4d7e-bba0-e2446fc96725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.796132 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b15df-52e9-458d-8c80-bb8442fdf5bd None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:57.804230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bac903-22ba-4d7e-bba0-e2446fc96725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:57.805778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bac903-22ba-4d7e-bba0-e2446fc96725 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:57.849599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ebcaa-ad97-4250-a1cf-f899f27424d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:57.850429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ebcaa-ad97-4250-a1cf-f899f27424d7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.851086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ebcaa-ad97-4250-a1cf-f899f27424d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.851373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ebcaa-ad97-4250-a1cf-f899f27424d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.865606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ebcaa-ad97-4250-a1cf-f899f27424d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-b12ebcaa-ad97-4250-a1cf-f899f27424d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:57.875718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:57.876572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:36:57.877395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 57/111] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:36:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:57.878377 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ebcaa-ad97-4250-a1cf-f899f27424d7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.886379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:57.913752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 55/112] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:36:57.920637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356a229-3a0a-463a-a930-2d0ca725cd35 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:57.920872 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356a229-3a0a-463a-a930-2d0ca725cd35 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:57.921154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356a229-3a0a-463a-a930-2d0ca725cd35 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:57.923189 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 56/113] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:57.923880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356a229-3a0a-463a-a930-2d0ca725cd35 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:57.943683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:57.996488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ebcaa-ad97-4250-a1cf-f899f27424d7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.031269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356a229-3a0a-463a-a930-2d0ca725cd35 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.038880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfd593-5d01-4ee0-bd31-a9c19d634004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.039733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfd593-5d01-4ee0-bd31-a9c19d634004 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.040080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfd593-5d01-4ee0-bd31-a9c19d634004 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.040367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfd593-5d01-4ee0-bd31-a9c19d634004 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.050730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfd593-5d01-4ee0-bd31-a9c19d634004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-76bfd593-5d01-4ee0-bd31-a9c19d634004 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:58.058605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:58.059672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:36:58.060659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 58/114] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:36:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:58.065948 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da467148-d2dc-44c5-82c7-d154d2e54cb1 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.066244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da467148-d2dc-44c5-82c7-d154d2e54cb1 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.066526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da467148-d2dc-44c5-82c7-d154d2e54cb1 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.068384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 59/115] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:58.070058 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da467148-d2dc-44c5-82c7-d154d2e54cb1 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.071722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7afa43-efc4-430f-977a-5b005c51d2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.072739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afa43-efc4-430f-977a-5b005c51d2f5 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.073223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afa43-efc4-430f-977a-5b005c51d2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.073745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7afa43-efc4-430f-977a-5b005c51d2f5 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.074725 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7afa43-efc4-430f-977a-5b005c51d2f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:58.075248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7afa43-efc4-430f-977a-5b005c51d2f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:58.076821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7afa43-efc4-430f-977a-5b005c51d2f5 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:58.086225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 57/116] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:36:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:58.087774 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932778-fbd1-40ce-9f23-f4eea7a4f56d None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.088166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932778-fbd1-40ce-9f23-f4eea7a4f56d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.088548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932778-fbd1-40ce-9f23-f4eea7a4f56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.105630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-8b932778-fbd1-40ce-9f23-f4eea7a4f56d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:58.106936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8b932778-fbd1-40ce-9f23-f4eea7a4f56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:58.126179 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:58.179544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 58/117] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:58.180690 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b932778-fbd1-40ce-9f23-f4eea7a4f56d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.193895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da467148-d2dc-44c5-82c7-d154d2e54cb1 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.210775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:58.236975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed664fcf-3727-460d-9250-ed0e8cd445f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.237919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed664fcf-3727-460d-9250-ed0e8cd445f1 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.238200 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed664fcf-3727-460d-9250-ed0e8cd445f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.238481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed664fcf-3727-460d-9250-ed0e8cd445f1 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.240211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed664fcf-3727-460d-9250-ed0e8cd445f1 None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:58.241713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed664fcf-3727-460d-9250-ed0e8cd445f1 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:58.243722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 60/118] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:58.245057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd7b35-0239-41e6-af01-be25b8123683 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.245375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd7b35-0239-41e6-af01-be25b8123683 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.245659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd7b35-0239-41e6-af01-be25b8123683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.261901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-febd7b35-0239-41e6-af01-be25b8123683 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:58.263090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-febd7b35-0239-41e6-af01-be25b8123683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:36:58.300564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b932778-fbd1-40ce-9f23-f4eea7a4f56d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.333294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 61/119] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:36:58.333294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd7b35-0239-41e6-af01-be25b8123683 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.350153 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e61b2c-cd8c-4856-aaa1-ec86ff86db94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.350846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e61b2c-cd8c-4856-aaa1-ec86ff86db94 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.351126 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e61b2c-cd8c-4856-aaa1-ec86ff86db94 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.351406 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e61b2c-cd8c-4856-aaa1-ec86ff86db94 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.352162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e61b2c-cd8c-4856-aaa1-ec86ff86db94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:58.352474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e61b2c-cd8c-4856-aaa1-ec86ff86db94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:58.354008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e61b2c-cd8c-4856-aaa1-ec86ff86db94 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:58.363773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 59/120] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:58.366463 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e61b2c-cd8c-4856-aaa1-ec86ff86db94 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.386697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:58.450745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd7b35-0239-41e6-af01-be25b8123683 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.457756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:58.482324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e61b2c-cd8c-4856-aaa1-ec86ff86db94 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.486342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797b778-1d7c-4ac0-8906-c1095d2cadb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.487677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b778-1d7c-4ac0-8906-c1095d2cadb4 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.487970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b778-1d7c-4ac0-8906-c1095d2cadb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.488250 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b778-1d7c-4ac0-8906-c1095d2cadb4 None admin] PATH_INFO: `/v3/users/cf1ee67d28b6436fa3ec26eeae38390a` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.490000 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797b778-1d7c-4ac0-8906-c1095d2cadb4 None admin] RBAC: Authorizing `identity:get_user(user_id=cf1ee67d28b6436fa3ec26eeae38390a)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:58.491465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797b778-1d7c-4ac0-8906-c1095d2cadb4 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:58.493327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 62/121] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/users/cf1ee67d28b6436fa3ec26eeae38390a => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:58.493902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797b778-1d7c-4ac0-8906-c1095d2cadb4 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.523567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d29530-66d7-4424-b63e-7e058c503239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.524714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d29530-66d7-4424-b63e-7e058c503239 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.525000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d29530-66d7-4424-b63e-7e058c503239 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.525310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d29530-66d7-4424-b63e-7e058c503239 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.535862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d29530-66d7-4424-b63e-7e058c503239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-a3d29530-66d7-4424-b63e-7e058c503239 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:58.544983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:58.545846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:36:58.546624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 60/122] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:58.547244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d29530-66d7-4424-b63e-7e058c503239 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.603625 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797b778-1d7c-4ac0-8906-c1095d2cadb4 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.634683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:58.638279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dedf3f-6088-4158-8319-145da332fcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.638952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dedf3f-6088-4158-8319-145da332fcda None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.639294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dedf3f-6088-4158-8319-145da332fcda None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.639576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dedf3f-6088-4158-8319-145da332fcda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.640411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dedf3f-6088-4158-8319-145da332fcda None admin] RBAC: Adding query filter params (role.id=43a16638caac4dd8acae2987c010b48a, scope.project.id=0e628770474b423e99809dd568e34a0e, user.id=3628f92de2bf465d90beaeb1f935f0c7) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:58.640725 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dedf3f-6088-4158-8319-145da332fcda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:58.642203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dedf3f-6088-4158-8319-145da332fcda None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:58.648733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 63/123] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/role_assignments?user.id=3628f92de2bf465d90beaeb1f935f0c7&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=43a16638caac4dd8acae2987c010b48a => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:58.649455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dedf3f-6088-4158-8319-145da332fcda None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.659374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d29530-66d7-4424-b63e-7e058c503239 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.703693 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfa98e-57e6-493c-a0c2-92976b4aeaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.704551 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfa98e-57e6-493c-a0c2-92976b4aeaa5 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.704893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfa98e-57e6-493c-a0c2-92976b4aeaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.705181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfa98e-57e6-493c-a0c2-92976b4aeaa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.718312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfa98e-57e6-493c-a0c2-92976b4aeaa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:36:58.719316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-7bcfa98e-57e6-493c-a0c2-92976b4aeaa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:36:58.728453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:36:58.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:36:58.730438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:36:58.732765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 61/124] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:36:58.734025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfa98e-57e6-493c-a0c2-92976b4aeaa5 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.804147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dedf3f-6088-4158-8319-145da332fcda None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.845940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf1c3a-021b-4e57-afcb-65ae4b817c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.847382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf1c3a-021b-4e57-afcb-65ae4b817c20 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.848073 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf1c3a-021b-4e57-afcb-65ae4b817c20 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.849405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf1c3a-021b-4e57-afcb-65ae4b817c20 None admin] PATH_INFO: `/v3/users/88fee257b59d48f2b418ffa9e86b6f8e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.851170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf1c3a-021b-4e57-afcb-65ae4b817c20 None admin] RBAC: Authorizing `identity:get_user(user_id=88fee257b59d48f2b418ffa9e86b6f8e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:58.852655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf1c3a-021b-4e57-afcb-65ae4b817c20 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:58.854525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 64/125] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/users/88fee257b59d48f2b418ffa9e86b6f8e => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:58.855164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf1c3a-021b-4e57-afcb-65ae4b817c20 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.856161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfa98e-57e6-493c-a0c2-92976b4aeaa5 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.900407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7d1dd-a2a2-459e-861a-383c865539c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:58.902174 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7d1dd-a2a2-459e-861a-383c865539c3 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:58.902841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7d1dd-a2a2-459e-861a-383c865539c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:58.903451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7d1dd-a2a2-459e-861a-383c865539c3 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:58.905042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7d1dd-a2a2-459e-861a-383c865539c3 None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:58.906977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7d1dd-a2a2-459e-861a-383c865539c3 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:58.909954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 62/126] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:58.911722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7d1dd-a2a2-459e-861a-383c865539c3 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:58.916916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:58.958686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf1c3a-021b-4e57-afcb-65ae4b817c20 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:58.993623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:58.999477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dfd6c-c35c-42e8-8a2e-db18e57e1a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.000291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dfd6c-c35c-42e8-8a2e-db18e57e1a86 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.000904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dfd6c-c35c-42e8-8a2e-db18e57e1a86 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.001314 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dfd6c-c35c-42e8-8a2e-db18e57e1a86 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.002668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3dfd6c-c35c-42e8-8a2e-db18e57e1a86 None admin] RBAC: Adding query filter params (name=member) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:59.003078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3dfd6c-c35c-42e8-8a2e-db18e57e1a86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.004817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3dfd6c-c35c-42e8-8a2e-db18e57e1a86 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.016344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 65/127] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:59.017276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dfd6c-c35c-42e8-8a2e-db18e57e1a86 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:59.050033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7d1dd-a2a2-459e-861a-383c865539c3 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.086390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69526c2-9f5c-4cd5-8128-b828b2bc7856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.087329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69526c2-9f5c-4cd5-8128-b828b2bc7856 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.087760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69526c2-9f5c-4cd5-8128-b828b2bc7856 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.088182 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69526c2-9f5c-4cd5-8128-b828b2bc7856 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.089064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69526c2-9f5c-4cd5-8128-b828b2bc7856 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:36:59.089562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69526c2-9f5c-4cd5-8128-b828b2bc7856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.091139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69526c2-9f5c-4cd5-8128-b828b2bc7856 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.100280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 63/128] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:36:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:59.101446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69526c2-9f5c-4cd5-8128-b828b2bc7856 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:59.157887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dfd6c-c35c-42e8-8a2e-db18e57e1a86 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.172610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:59.205781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310e228-9431-4172-917f-5518465c4843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.207388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310e228-9431-4172-917f-5518465c4843 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.207687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310e228-9431-4172-917f-5518465c4843 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.207973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310e228-9431-4172-917f-5518465c4843 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.210011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310e228-9431-4172-917f-5518465c4843 None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.211449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310e228-9431-4172-917f-5518465c4843 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.211701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69526c2-9f5c-4cd5-8128-b828b2bc7856 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.213201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 66/129] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:36:59 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:59.214273 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310e228-9431-4172-917f-5518465c4843 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:59.254363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c78cc-5eed-4db1-a4e8-f500849e48aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.255836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c78cc-5eed-4db1-a4e8-f500849e48aa None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.256128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c78cc-5eed-4db1-a4e8-f500849e48aa None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.256414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c78cc-5eed-4db1-a4e8-f500849e48aa None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e/users/cf1ee67d28b6436fa3ec26eeae38390a/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.258621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:59.260793 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c78cc-5eed-4db1-a4e8-f500849e48aa None admin] RBAC: Authorizing `identity:create_grant(project_id=0e628770474b423e99809dd568e34a0e, user_id=cf1ee67d28b6436fa3ec26eeae38390a, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.262534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c78cc-5eed-4db1-a4e8-f500849e48aa None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.325142 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310e228-9431-4172-917f-5518465c4843 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.357605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 64/130] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:36:59 2022] PUT /identity/v3/projects/0e628770474b423e99809dd568e34a0e/users/cf1ee67d28b6436fa3ec26eeae38390a/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:59.362935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c78cc-5eed-4db1-a4e8-f500849e48aa None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:59.369712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58406b-a6a1-4720-bea6-dd08988850c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.370665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58406b-a6a1-4720-bea6-dd08988850c5 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.370960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58406b-a6a1-4720-bea6-dd08988850c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.371298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58406b-a6a1-4720-bea6-dd08988850c5 None admin] PATH_INFO: `/v3/users/cf1ee67d28b6436fa3ec26eeae38390a` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.372699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58406b-a6a1-4720-bea6-dd08988850c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cf1ee67d28b6436fa3ec26eeae38390a)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.374327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58406b-a6a1-4720-bea6-dd08988850c5 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.376253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 67/131] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:59 2022] GET /identity/v3/users/cf1ee67d28b6436fa3ec26eeae38390a => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:59.381394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58406b-a6a1-4720-bea6-dd08988850c5 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:59.555046 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c78cc-5eed-4db1-a4e8-f500849e48aa None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.568155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58406b-a6a1-4720-bea6-dd08988850c5 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.577634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:59.598276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:59.601518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c785a4-5838-48c0-bb83-c621130e3196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.602741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c785a4-5838-48c0-bb83-c621130e3196 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.603101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c785a4-5838-48c0-bb83-c621130e3196 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.603445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c785a4-5838-48c0-bb83-c621130e3196 None admin] PATH_INFO: `/v3/users/cf1ee67d28b6436fa3ec26eeae38390a` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.605338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c785a4-5838-48c0-bb83-c621130e3196 None admin] RBAC: Authorizing `identity:get_user(user_id=cf1ee67d28b6436fa3ec26eeae38390a)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.607301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c785a4-5838-48c0-bb83-c621130e3196 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.613268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 65/132] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:36:59 2022] GET /identity/v3/users/cf1ee67d28b6436fa3ec26eeae38390a => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:59.613268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c785a4-5838-48c0-bb83-c621130e3196 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:59.618014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c77802-e5c4-47c8-b9c6-c17f24b459b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.619256 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c77802-e5c4-47c8-b9c6-c17f24b459b5 None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.619730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c77802-e5c4-47c8-b9c6-c17f24b459b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.620191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c77802-e5c4-47c8-b9c6-c17f24b459b5 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e/users/88fee257b59d48f2b418ffa9e86b6f8e/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.622742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c77802-e5c4-47c8-b9c6-c17f24b459b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e628770474b423e99809dd568e34a0e, user_id=88fee257b59d48f2b418ffa9e86b6f8e, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.624665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c77802-e5c4-47c8-b9c6-c17f24b459b5 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.713634 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c785a4-5838-48c0-bb83-c621130e3196 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.732220 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 68/133] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:36:59 2022] PUT /identity/v3/projects/0e628770474b423e99809dd568e34a0e/users/88fee257b59d48f2b418ffa9e86b6f8e/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:36:59.733134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c77802-e5c4-47c8-b9c6-c17f24b459b5 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:59.749190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66908dfc-1647-4fc9-8a0c-a67421338216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.750183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908dfc-1647-4fc9-8a0c-a67421338216 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.750477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908dfc-1647-4fc9-8a0c-a67421338216 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.750756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66908dfc-1647-4fc9-8a0c-a67421338216 None admin] PATH_INFO: `/v3/projects/78876b2fbff64f139ae1172647ebbc4e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.758409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66908dfc-1647-4fc9-8a0c-a67421338216 None admin] RBAC: Authorizing `identity:get_project(project_id=78876b2fbff64f139ae1172647ebbc4e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.760134 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66908dfc-1647-4fc9-8a0c-a67421338216 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.762081 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 66/134] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:36:59 2022] GET /identity/v3/projects/78876b2fbff64f139ae1172647ebbc4e => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:59.764618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79059b-544c-46c8-b2aa-a6db64acaf0e None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.764618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79059b-544c-46c8-b2aa-a6db64acaf0e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.764618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a79059b-544c-46c8-b2aa-a6db64acaf0e None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.765353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 67/135] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:36:59.770298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79059b-544c-46c8-b2aa-a6db64acaf0e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:36:59.845774 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:36:59.893375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a79059b-544c-46c8-b2aa-a6db64acaf0e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.895394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c77802-e5c4-47c8-b9c6-c17f24b459b5 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:36:59.933772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3921a-1fd0-4d44-8c07-999a03894c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.934731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3921a-1fd0-4d44-8c07-999a03894c26 None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.935024 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3921a-1fd0-4d44-8c07-999a03894c26 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.935303 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3921a-1fd0-4d44-8c07-999a03894c26 None admin] PATH_INFO: `/v3/projects/78876b2fbff64f139ae1172647ebbc4e/users/cf1ee67d28b6436fa3ec26eeae38390a/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.937363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3921a-1fd0-4d44-8c07-999a03894c26 None admin] RBAC: Authorizing `identity:create_grant(project_id=78876b2fbff64f139ae1172647ebbc4e, user_id=cf1ee67d28b6436fa3ec26eeae38390a, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.939015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb3921a-1fd0-4d44-8c07-999a03894c26 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.961511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186c51d-3eb1-48dd-abbb-c7e8fd56078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:36:59.962466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186c51d-3eb1-48dd-abbb-c7e8fd56078a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.962764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186c51d-3eb1-48dd-abbb-c7e8fd56078a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.963053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186c51d-3eb1-48dd-abbb-c7e8fd56078a None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.964402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186c51d-3eb1-48dd-abbb-c7e8fd56078a None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:36:59.965995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186c51d-3eb1-48dd-abbb-c7e8fd56078a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:36:59.967896 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 69/136] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:36:59 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:36:59.969199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47a49a-0322-4aae-b293-6bfde642a480 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:36:59.969551 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47a49a-0322-4aae-b293-6bfde642a480 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:36:59.969840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47a49a-0322-4aae-b293-6bfde642a480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:36:59.989516 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-3b47a49a-0322-4aae-b293-6bfde642a480 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:36:59.990906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-3b47a49a-0322-4aae-b293-6bfde642a480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:00.020356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:00.066731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 70/137] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:00.066928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 68/138] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:36:59 2022] PUT /identity/v3/projects/78876b2fbff64f139ae1172647ebbc4e/users/cf1ee67d28b6436fa3ec26eeae38390a/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:00.067808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3921a-1fd0-4d44-8c07-999a03894c26 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:00.118030 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47a49a-0322-4aae-b293-6bfde642a480 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:00.243677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3921a-1fd0-4d44-8c07-999a03894c26 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:00.293610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:00.321604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398a745-8b3e-413e-8011-10a79a1c4cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:00.322515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398a745-8b3e-413e-8011-10a79a1c4cd0 None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:00.322865 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398a745-8b3e-413e-8011-10a79a1c4cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:00.323154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398a745-8b3e-413e-8011-10a79a1c4cd0 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e/users/cf1ee67d28b6436fa3ec26eeae38390a/roles/5f24daf2ba2b4f8fab95c87cefb52231` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:00.331454 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398a745-8b3e-413e-8011-10a79a1c4cd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e628770474b423e99809dd568e34a0e, user_id=cf1ee67d28b6436fa3ec26eeae38390a, role_id=5f24daf2ba2b4f8fab95c87cefb52231)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:00.332932 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398a745-8b3e-413e-8011-10a79a1c4cd0 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:00.372876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b47a49a-0322-4aae-b293-6bfde642a480 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:00.422045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09def3-72f9-4d44-932e-e6911e0cf498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:00.422859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09def3-72f9-4d44-932e-e6911e0cf498 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:00.423148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09def3-72f9-4d44-932e-e6911e0cf498 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:00.423487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09def3-72f9-4d44-932e-e6911e0cf498 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:00.424725 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09def3-72f9-4d44-932e-e6911e0cf498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:00.426354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09def3-72f9-4d44-932e-e6911e0cf498 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:00.432191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 71/139] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:00.439983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 69/140] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:37:00 2022] PUT /identity/v3/projects/0e628770474b423e99809dd568e34a0e/users/cf1ee67d28b6436fa3ec26eeae38390a/roles/5f24daf2ba2b4f8fab95c87cefb52231 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:00.443140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09def3-72f9-4d44-932e-e6911e0cf498 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:00.477062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:00.566857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09def3-72f9-4d44-932e-e6911e0cf498 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:00.604826 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d4b4c-ddde-4cbf-83d9-5c389df5d6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:00.605536 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d4b4c-ddde-4cbf-83d9-5c389df5d6f9 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:00.605829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d4b4c-ddde-4cbf-83d9-5c389df5d6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:00.606115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d4b4c-ddde-4cbf-83d9-5c389df5d6f9 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:00.606839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d4b4c-ddde-4cbf-83d9-5c389df5d6f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:00.608311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d4b4c-ddde-4cbf-83d9-5c389df5d6f9 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:00.738447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 72/141] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:37:00 2022] POST /identity/v3/users => generated 283 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:01.201156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481c2c-6006-4960-a40c-5d865dbfadd7 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.201550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481c2c-6006-4960-a40c-5d865dbfadd7 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.201772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c481c2c-6006-4960-a40c-5d865dbfadd7 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.204883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 70/142] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:01.210285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43e63a-bd4a-400f-8dc5-0968416438dc None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.210588 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43e63a-bd4a-400f-8dc5-0968416438dc None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.210876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43e63a-bd4a-400f-8dc5-0968416438dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.233163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-bc43e63a-bd4a-400f-8dc5-0968416438dc None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:01.234399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-bc43e63a-bd4a-400f-8dc5-0968416438dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:01.321151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 73/143] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:01.355141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c481c2c-6006-4960-a40c-5d865dbfadd7 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:01.506582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c481c2c-6006-4960-a40c-5d865dbfadd7 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:01.545140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5d103-7928-4748-a8d7-46ceec86d3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:01.546003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5d103-7928-4748-a8d7-46ceec86d3c7 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.546297 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5d103-7928-4748-a8d7-46ceec86d3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.546580 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a5d103-7928-4748-a8d7-46ceec86d3c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.559977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a5d103-7928-4748-a8d7-46ceec86d3c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-74a5d103-7928-4748-a8d7-46ceec86d3c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:01.567701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:01.568575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:01.569401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 71/144] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:01.576607 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43e63a-bd4a-400f-8dc5-0968416438dc None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:01.605042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:01.700149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43e63a-bd4a-400f-8dc5-0968416438dc None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:01.740481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b47d4-58f7-4efa-94aa-24ce49e31dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:01.741168 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b47d4-58f7-4efa-94aa-24ce49e31dfa None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.741487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b47d4-58f7-4efa-94aa-24ce49e31dfa None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.741763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b47d4-58f7-4efa-94aa-24ce49e31dfa None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.742512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b47d4-58f7-4efa-94aa-24ce49e31dfa None admin] RBAC: Adding query filter params (name=reader) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:01.742819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b47d4-58f7-4efa-94aa-24ce49e31dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:01.744252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b47d4-58f7-4efa-94aa-24ce49e31dfa None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:01.754279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 74/145] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:01.760577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5d103-7928-4748-a8d7-46ceec86d3c7 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:01.806323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:01.865888 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a5d103-7928-4748-a8d7-46ceec86d3c7 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:01.914869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c044df-3534-45d5-b626-a8462b5fbe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:01.915782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c044df-3534-45d5-b626-a8462b5fbe57 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:01.916072 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c044df-3534-45d5-b626-a8462b5fbe57 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:01.916349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c044df-3534-45d5-b626-a8462b5fbe57 None admin] PATH_INFO: `/v3/users/3628f92de2bf465d90beaeb1f935f0c7` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:01.917944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c044df-3534-45d5-b626-a8462b5fbe57 None admin] RBAC: Authorizing `identity:get_user(user_id=3628f92de2bf465d90beaeb1f935f0c7)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:01.919400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c044df-3534-45d5-b626-a8462b5fbe57 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:01.921196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 72/146] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:01 2022] GET /identity/v3/users/3628f92de2bf465d90beaeb1f935f0c7 => generated 289 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:01.926619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b47d4-58f7-4efa-94aa-24ce49e31dfa None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:02.016342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:02.041301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b47d4-58f7-4efa-94aa-24ce49e31dfa None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:02.080656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bd69e-a0c8-4595-b81c-b4dda4ceba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:02.081549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bd69e-a0c8-4595-b81c-b4dda4ceba08 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:02.081852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bd69e-a0c8-4595-b81c-b4dda4ceba08 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:02.082130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bd69e-a0c8-4595-b81c-b4dda4ceba08 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:02.083338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bd69e-a0c8-4595-b81c-b4dda4ceba08 None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:02.085370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bd69e-a0c8-4595-b81c-b4dda4ceba08 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:02.087081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 75/147] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:01 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:02.092855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c044df-3534-45d5-b626-a8462b5fbe57 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:02.226519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c044df-3534-45d5-b626-a8462b5fbe57 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:02.241427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:02.266028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a00e40-4c6d-4604-abb2-3cb408686308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:02.266983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a00e40-4c6d-4604-abb2-3cb408686308 None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:02.267214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a00e40-4c6d-4604-abb2-3cb408686308 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:02.267498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a00e40-4c6d-4604-abb2-3cb408686308 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e/users/3628f92de2bf465d90beaeb1f935f0c7/roles/43a16638caac4dd8acae2987c010b48a` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:02.269536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a00e40-4c6d-4604-abb2-3cb408686308 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e628770474b423e99809dd568e34a0e, user_id=3628f92de2bf465d90beaeb1f935f0c7, role_id=43a16638caac4dd8acae2987c010b48a)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:02.271163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a00e40-4c6d-4604-abb2-3cb408686308 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:02.367612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 73/148] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:37:02 2022] PUT /identity/v3/projects/0e628770474b423e99809dd568e34a0e/users/3628f92de2bf465d90beaeb1f935f0c7/roles/43a16638caac4dd8acae2987c010b48a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:02.630041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff350d5-3785-4ea9-8901-04bfb4bfab58 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:02.631114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff350d5-3785-4ea9-8901-04bfb4bfab58 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:02.631114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff350d5-3785-4ea9-8901-04bfb4bfab58 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:02.632832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 76/149] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:02.639644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d85cc7-7227-417d-b5e2-b52b729465ce None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:02.640100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d85cc7-7227-417d-b5e2-b52b729465ce None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:02.640263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d85cc7-7227-417d-b5e2-b52b729465ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:02.660354 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-53d85cc7-7227-417d-b5e2-b52b729465ce None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:02.661597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-53d85cc7-7227-417d-b5e2-b52b729465ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:02.745643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 74/150] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:02.780530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff350d5-3785-4ea9-8901-04bfb4bfab58 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:02.892627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff350d5-3785-4ea9-8901-04bfb4bfab58 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:02.933676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c159905-371a-47ab-8d28-52f659eba319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:02.934485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c159905-371a-47ab-8d28-52f659eba319 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:02.934771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c159905-371a-47ab-8d28-52f659eba319 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:02.935051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c159905-371a-47ab-8d28-52f659eba319 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:02.950847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c159905-371a-47ab-8d28-52f659eba319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-9c159905-371a-47ab-8d28-52f659eba319 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:02.961160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:02.962091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:02.962842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 77/151] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:02.968431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d85cc7-7227-417d-b5e2-b52b729465ce None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.124560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:03.128884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d85cc7-7227-417d-b5e2-b52b729465ce None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.173500 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1faca2-10b5-4eec-8f53-037dfde861c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.174089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1faca2-10b5-4eec-8f53-037dfde861c7 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.174370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1faca2-10b5-4eec-8f53-037dfde861c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.174646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1faca2-10b5-4eec-8f53-037dfde861c7 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.175388 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1faca2-10b5-4eec-8f53-037dfde861c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:03.175695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1faca2-10b5-4eec-8f53-037dfde861c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.177080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1faca2-10b5-4eec-8f53-037dfde861c7 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.190673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 75/152] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:03.197381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c159905-371a-47ab-8d28-52f659eba319 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.212128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:03.238602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18f28c-daee-430b-ae79-7ed8755b6e52 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.238924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18f28c-daee-430b-ae79-7ed8755b6e52 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.239161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18f28c-daee-430b-ae79-7ed8755b6e52 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.241069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 76/153] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:37:03.246577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26ff1d-46c3-4852-ae33-d2a29a9af2f9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.246869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26ff1d-46c3-4852-ae33-d2a29a9af2f9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.247144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26ff1d-46c3-4852-ae33-d2a29a9af2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.274332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-dc26ff1d-46c3-4852-ae33-d2a29a9af2f9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:03.275537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-dc26ff1d-46c3-4852-ae33-d2a29a9af2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:03.391236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c159905-371a-47ab-8d28-52f659eba319 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.447570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 77/154] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:03.448852 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a47511-ea21-439e-ab89-cae7947c7619 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.449181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a47511-ea21-439e-ab89-cae7947c7619 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.449431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a47511-ea21-439e-ab89-cae7947c7619 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.451027 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 78/155] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:03.459045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39e3bf-7dea-4015-8793-43152d5776ff None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.459330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39e3bf-7dea-4015-8793-43152d5776ff None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.459608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39e3bf-7dea-4015-8793-43152d5776ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.472369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f034b28-2526-4589-831c-b3ca7db95f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.473267 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f034b28-2526-4589-831c-b3ca7db95f2a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.473565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f034b28-2526-4589-831c-b3ca7db95f2a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.473840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f034b28-2526-4589-831c-b3ca7db95f2a None admin] PATH_INFO: `/v3/users/cf1ee67d28b6436fa3ec26eeae38390a` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.475604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f034b28-2526-4589-831c-b3ca7db95f2a None admin] RBAC: Authorizing `identity:get_user(user_id=cf1ee67d28b6436fa3ec26eeae38390a)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.477078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f034b28-2526-4589-831c-b3ca7db95f2a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.483150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 78/156] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:03 2022] GET /identity/v3/users/cf1ee67d28b6436fa3ec26eeae38390a => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:03.488854 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f034b28-2526-4589-831c-b3ca7db95f2a None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.493018 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6c39e3bf-7dea-4015-8793-43152d5776ff None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:03.494338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6c39e3bf-7dea-4015-8793-43152d5776ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:03.633040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 79/157] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:03.634095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39e3bf-7dea-4015-8793-43152d5776ff None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:03.650175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:03.681041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:03.707428 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f034b28-2526-4589-831c-b3ca7db95f2a None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.769979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b51642-b2cb-4e0d-8f50-879cb1014fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.771794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b51642-b2cb-4e0d-8f50-879cb1014fb8 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.772110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b51642-b2cb-4e0d-8f50-879cb1014fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.772356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b51642-b2cb-4e0d-8f50-879cb1014fb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.873107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39e3bf-7dea-4015-8793-43152d5776ff None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:03.882481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b51642-b2cb-4e0d-8f50-879cb1014fb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:03.883982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b51642-b2cb-4e0d-8f50-879cb1014fb8 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:03.947634 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab796f-27f5-4d2b-826c-7886a3c87b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:03.948440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab796f-27f5-4d2b-826c-7886a3c87b79 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:03.948722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab796f-27f5-4d2b-826c-7886a3c87b79 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:03.948997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab796f-27f5-4d2b-826c-7886a3c87b79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:03.966119 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab796f-27f5-4d2b-826c-7886a3c87b79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-25ab796f-27f5-4d2b-826c-7886a3c87b79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:03.973854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:03.975374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 80/158] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:03.976547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab796f-27f5-4d2b-826c-7886a3c87b79 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.001653 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:04.053258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 79/159] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:04.060648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d80a95-1cc6-4f97-9f7e-fa011d7bcda1 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.060950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d80a95-1cc6-4f97-9f7e-fa011d7bcda1 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.061217 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d80a95-1cc6-4f97-9f7e-fa011d7bcda1 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.062682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 80/160] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:04.063445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d80a95-1cc6-4f97-9f7e-fa011d7bcda1 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.203533 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:04.241173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab796f-27f5-4d2b-826c-7886a3c87b79 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:04.261259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d80a95-1cc6-4f97-9f7e-fa011d7bcda1 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:04.320901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e85fa-2d0c-4607-984a-ec6cf2decc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:04.321615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e85fa-2d0c-4607-984a-ec6cf2decc0e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.321909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e85fa-2d0c-4607-984a-ec6cf2decc0e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.322190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e85fa-2d0c-4607-984a-ec6cf2decc0e None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.323008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e85fa-2d0c-4607-984a-ec6cf2decc0e None admin] RBAC: Adding query filter params (name=member) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:04.323318 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e85fa-2d0c-4607-984a-ec6cf2decc0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:04.324848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e85fa-2d0c-4607-984a-ec6cf2decc0e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:04.339100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56cadb-329f-4368-986c-528f63e227ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:04.339931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56cadb-329f-4368-986c-528f63e227ba None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.340220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56cadb-329f-4368-986c-528f63e227ba None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.340503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd56cadb-329f-4368-986c-528f63e227ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.342062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 81/161] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:04.345742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcb247-af34-4e27-adad-c375386d2a31 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.346314 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcb247-af34-4e27-adad-c375386d2a31 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.346808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcb247-af34-4e27-adad-c375386d2a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.377911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd56cadb-329f-4368-986c-528f63e227ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:04.379634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f5bcb247-af34-4e27-adad-c375386d2a31 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:04.380908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f5bcb247-af34-4e27-adad-c375386d2a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-cd56cadb-329f-4368-986c-528f63e227ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:04.390061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:04.390965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:04.391724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 81/162] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:37:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:04.392911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56cadb-329f-4368-986c-528f63e227ba None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.433611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:04.507191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 82/163] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:04.507946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcb247-af34-4e27-adad-c375386d2a31 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.573561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd56cadb-329f-4368-986c-528f63e227ba None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:04.651796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50d390-a650-4c99-a4e7-63cb381bb96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:04.652744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcb247-af34-4e27-adad-c375386d2a31 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:04.653682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50d390-a650-4c99-a4e7-63cb381bb96e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.654268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50d390-a650-4c99-a4e7-63cb381bb96e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.654740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d50d390-a650-4c99-a4e7-63cb381bb96e None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.661575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50d390-a650-4c99-a4e7-63cb381bb96e None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:04.663956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d50d390-a650-4c99-a4e7-63cb381bb96e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:04.666374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 82/164] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:04 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:04.667379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50d390-a650-4c99-a4e7-63cb381bb96e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.690098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:04.696826 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6f0b8-7d5d-4f2f-9c79-da85e51713f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:04.697799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6f0b8-7d5d-4f2f-9c79-da85e51713f1 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.698386 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6f0b8-7d5d-4f2f-9c79-da85e51713f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.698386 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6f0b8-7d5d-4f2f-9c79-da85e51713f1 None admin] PATH_INFO: `/v3/users/cf1ee67d28b6436fa3ec26eeae38390a` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.700796 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6f0b8-7d5d-4f2f-9c79-da85e51713f1 None admin] RBAC: Authorizing `identity:get_user(user_id=cf1ee67d28b6436fa3ec26eeae38390a)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:04.702411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6f0b8-7d5d-4f2f-9c79-da85e51713f1 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:04.704448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 83/165] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:04 2022] GET /identity/v3/users/cf1ee67d28b6436fa3ec26eeae38390a => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:04.709076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6f0b8-7d5d-4f2f-9c79-da85e51713f1 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.796232 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:04.889320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d50d390-a650-4c99-a4e7-63cb381bb96e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:04.925422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d673a0c-b336-491f-a589-e3603da4f6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:04.926441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d673a0c-b336-491f-a589-e3603da4f6e9 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:04.926939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d673a0c-b336-491f-a589-e3603da4f6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:04.927465 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d673a0c-b336-491f-a589-e3603da4f6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:04.928420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d673a0c-b336-491f-a589-e3603da4f6e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:04.928955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d673a0c-b336-491f-a589-e3603da4f6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:04.930562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d673a0c-b336-491f-a589-e3603da4f6e9 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:04.939312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 83/166] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:04.940339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d673a0c-b336-491f-a589-e3603da4f6e9 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:04.967387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6f0b8-7d5d-4f2f-9c79-da85e51713f1 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:05.019013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:05.052679 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d673a0c-b336-491f-a589-e3603da4f6e9 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:05.061940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b495a87-a168-46c9-8ad5-f463682202bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:05.062778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b495a87-a168-46c9-8ad5-f463682202bb None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:05.063067 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b495a87-a168-46c9-8ad5-f463682202bb None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:05.063352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b495a87-a168-46c9-8ad5-f463682202bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:05.090393 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b90f7-c5f6-47be-859d-b09764121452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:05.090606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b495a87-a168-46c9-8ad5-f463682202bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:05.091511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b90f7-c5f6-47be-859d-b09764121452 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:05.092015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b90f7-c5f6-47be-859d-b09764121452 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:05.092497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b90f7-c5f6-47be-859d-b09764121452 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:05.093581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b90f7-c5f6-47be-859d-b09764121452 None admin] RBAC: Adding query filter params (role.id=0960434fd9da404ea96841eb4306eaae, scope.project.id=0e628770474b423e99809dd568e34a0e, user.id=cf1ee67d28b6436fa3ec26eeae38390a) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:05.094139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b90f7-c5f6-47be-859d-b09764121452 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:05.095870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b90f7-c5f6-47be-859d-b09764121452 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-6b495a87-a168-46c9-8ad5-f463682202bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:05.098710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:05.099593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:05.100487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 84/167] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:37:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:05.105543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 84/168] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:04 2022] GET /identity/v3/role_assignments?user.id=cf1ee67d28b6436fa3ec26eeae38390a&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=0960434fd9da404ea96841eb4306eaae => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:05.106566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b90f7-c5f6-47be-859d-b09764121452 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:05.110886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b495a87-a168-46c9-8ad5-f463682202bb None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:05.256567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b90f7-c5f6-47be-859d-b09764121452 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:05.293706 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca31586-b50f-4226-bc3f-4161b2bf750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:05.294978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca31586-b50f-4226-bc3f-4161b2bf750b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:05.295481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca31586-b50f-4226-bc3f-4161b2bf750b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:05.295986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca31586-b50f-4226-bc3f-4161b2bf750b None admin] PATH_INFO: `/v3/users/88fee257b59d48f2b418ffa9e86b6f8e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:05.297450 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca31586-b50f-4226-bc3f-4161b2bf750b None admin] RBAC: Authorizing `identity:get_user(user_id=88fee257b59d48f2b418ffa9e86b6f8e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:05.299165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca31586-b50f-4226-bc3f-4161b2bf750b None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:05.301089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 85/169] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:05 2022] GET /identity/v3/users/88fee257b59d48f2b418ffa9e86b6f8e => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:05.302113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca31586-b50f-4226-bc3f-4161b2bf750b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:05.309581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:05.319724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b495a87-a168-46c9-8ad5-f463682202bb None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:05.321823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:05.383051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dce4a-bb7c-48bc-b32e-a41c8d9e3d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:05.383739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dce4a-bb7c-48bc-b32e-a41c8d9e3d0c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:05.384029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dce4a-bb7c-48bc-b32e-a41c8d9e3d0c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:05.384317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dce4a-bb7c-48bc-b32e-a41c8d9e3d0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:05.420173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca31586-b50f-4226-bc3f-4161b2bf750b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:05.458495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de4a31-5dfc-4802-bac3-207b8d67fcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:05.459352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de4a31-5dfc-4802-bac3-207b8d67fcf1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:05.459677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de4a31-5dfc-4802-bac3-207b8d67fcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:05.460024 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de4a31-5dfc-4802-bac3-207b8d67fcf1 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:05.460778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de4a31-5dfc-4802-bac3-207b8d67fcf1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:05.461089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de4a31-5dfc-4802-bac3-207b8d67fcf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:05.462595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de4a31-5dfc-4802-bac3-207b8d67fcf1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:05.471704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 86/170] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:05.472809 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de4a31-5dfc-4802-bac3-207b8d67fcf1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:05.497046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dce4a-bb7c-48bc-b32e-a41c8d9e3d0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:05.498577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dce4a-bb7c-48bc-b32e-a41c8d9e3d0c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:05.553687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:05.586891 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de4a31-5dfc-4802-bac3-207b8d67fcf1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:05.625016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c53d4d-d3c7-44a1-9875-4d3134d06433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:05.625710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c53d4d-d3c7-44a1-9875-4d3134d06433 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:05.626003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c53d4d-d3c7-44a1-9875-4d3134d06433 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:05.626286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c53d4d-d3c7-44a1-9875-4d3134d06433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:05.637008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:05.649211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 85/171] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:05.650087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dce4a-bb7c-48bc-b32e-a41c8d9e3d0c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:05.709214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c53d4d-d3c7-44a1-9875-4d3134d06433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:05.710926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c53d4d-d3c7-44a1-9875-4d3134d06433 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:05.785497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dce4a-bb7c-48bc-b32e-a41c8d9e3d0c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:05.786310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:05.821351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 87/172] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:05.822157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c53d4d-d3c7-44a1-9875-4d3134d06433 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:05.827255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85586e-9424-42ea-8a6b-1634259f4b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:05.828169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85586e-9424-42ea-8a6b-1634259f4b0c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:05.828446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85586e-9424-42ea-8a6b-1634259f4b0c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:05.828726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85586e-9424-42ea-8a6b-1634259f4b0c None admin] PATH_INFO: `/v3/users/cf1ee67d28b6436fa3ec26eeae38390a` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:05.829974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85586e-9424-42ea-8a6b-1634259f4b0c None admin] RBAC: Authorizing `identity:get_user(user_id=cf1ee67d28b6436fa3ec26eeae38390a)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:05.831452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85586e-9424-42ea-8a6b-1634259f4b0c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:05.833122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 86/173] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:05 2022] GET /identity/v3/users/cf1ee67d28b6436fa3ec26eeae38390a => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:05.840230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85586e-9424-42ea-8a6b-1634259f4b0c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:05.937181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c53d4d-d3c7-44a1-9875-4d3134d06433 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:05.943093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:05.978161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188617c3-fa36-4458-9d28-6e7f58f012c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:05.979278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188617c3-fa36-4458-9d28-6e7f58f012c3 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:05.979744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188617c3-fa36-4458-9d28-6e7f58f012c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:05.980165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188617c3-fa36-4458-9d28-6e7f58f012c3 None admin] PATH_INFO: `/v3/projects/78876b2fbff64f139ae1172647ebbc4e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:05.981698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188617c3-fa36-4458-9d28-6e7f58f012c3 None admin] RBAC: Authorizing `identity:get_project(project_id=78876b2fbff64f139ae1172647ebbc4e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:05.984364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188617c3-fa36-4458-9d28-6e7f58f012c3 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:05.986617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 88/174] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:05 2022] GET /identity/v3/projects/78876b2fbff64f139ae1172647ebbc4e => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:05.987641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188617c3-fa36-4458-9d28-6e7f58f012c3 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:06.030135 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:06.088099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85586e-9424-42ea-8a6b-1634259f4b0c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:06.104932 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188617c3-fa36-4458-9d28-6e7f58f012c3 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:06.123325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb326626-f51c-43e2-946f-0887eaa2d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:06.124385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb326626-f51c-43e2-946f-0887eaa2d338 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.124842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb326626-f51c-43e2-946f-0887eaa2d338 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.125146 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb326626-f51c-43e2-946f-0887eaa2d338 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.126623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb326626-f51c-43e2-946f-0887eaa2d338 None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:06.128264 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb326626-f51c-43e2-946f-0887eaa2d338 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:06.130644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 87/175] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:05 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:06.132179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb326626-f51c-43e2-946f-0887eaa2d338 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:06.148089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a616a-d8d5-4298-bd5e-47e64041330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:06.149149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a616a-d8d5-4298-bd5e-47e64041330c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.149659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a616a-d8d5-4298-bd5e-47e64041330c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.150248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a616a-d8d5-4298-bd5e-47e64041330c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.228091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a616a-d8d5-4298-bd5e-47e64041330c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:06.229730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a616a-d8d5-4298-bd5e-47e64041330c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:06.241108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb326626-f51c-43e2-946f-0887eaa2d338 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:06.266622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:06.270077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:06.276558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecfd47-c75c-42e5-8189-45830465ba5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:06.276982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecfd47-c75c-42e5-8189-45830465ba5f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.277351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecfd47-c75c-42e5-8189-45830465ba5f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.277628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecfd47-c75c-42e5-8189-45830465ba5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.278445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecfd47-c75c-42e5-8189-45830465ba5f None admin] RBAC: Adding query filter params (role.id=0960434fd9da404ea96841eb4306eaae, scope.project.id=78876b2fbff64f139ae1172647ebbc4e, user.id=cf1ee67d28b6436fa3ec26eeae38390a) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:06.279463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecfd47-c75c-42e5-8189-45830465ba5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:06.280433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ecfd47-c75c-42e5-8189-45830465ba5f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:06.286783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 88/176] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:06 2022] GET /identity/v3/role_assignments?user.id=cf1ee67d28b6436fa3ec26eeae38390a&scope.project.id=78876b2fbff64f139ae1172647ebbc4e&role.id=0960434fd9da404ea96841eb4306eaae => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:06.294905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b7774-cbd4-42f7-9416-1787f8ab1662 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.295241 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b7774-cbd4-42f7-9416-1787f8ab1662 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.295458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b7774-cbd4-42f7-9416-1787f8ab1662 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.296978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 89/177] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:06.298361 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6781-0eeb-4b0f-a08e-7f52a1a5c8b0 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.298595 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6781-0eeb-4b0f-a08e-7f52a1a5c8b0 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.298942 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6781-0eeb-4b0f-a08e-7f52a1a5c8b0 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.303971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 90/178] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:06.303971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc6781-0eeb-4b0f-a08e-7f52a1a5c8b0 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:06.358261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 89/179] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:06.359930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f34eeb-a305-4afa-a2c5-cdc57f5d9bcf None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.360408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f34eeb-a305-4afa-a2c5-cdc57f5d9bcf None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.360883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f34eeb-a305-4afa-a2c5-cdc57f5d9bcf None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.363042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 90/180] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:06.364380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6258d-fd2c-4bf7-9120-e948cd813dcc None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.364884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6258d-fd2c-4bf7-9120-e948cd813dcc None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.365937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6258d-fd2c-4bf7-9120-e948cd813dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.386030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d4c6258d-fd2c-4bf7-9120-e948cd813dcc None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:06.387288 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d4c6258d-fd2c-4bf7-9120-e948cd813dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:06.435745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc6781-0eeb-4b0f-a08e-7f52a1a5c8b0 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:06.463390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 91/181] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:06.464687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36920f4-f138-443d-aa2d-31df5d436904 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.464977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36920f4-f138-443d-aa2d-31df5d436904 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.465270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36920f4-f138-443d-aa2d-31df5d436904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.473915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847a150-c9e8-4e3a-bbb7-ec3e051bbb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:06.475115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847a150-c9e8-4e3a-bbb7-ec3e051bbb30 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.475405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847a150-c9e8-4e3a-bbb7-ec3e051bbb30 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.475680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847a150-c9e8-4e3a-bbb7-ec3e051bbb30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.476513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847a150-c9e8-4e3a-bbb7-ec3e051bbb30 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=0e628770474b423e99809dd568e34a0e, user.id=88fee257b59d48f2b418ffa9e86b6f8e) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:06.476824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847a150-c9e8-4e3a-bbb7-ec3e051bbb30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:06.478769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847a150-c9e8-4e3a-bbb7-ec3e051bbb30 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:06.485386 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 91/182] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:06 2022] GET /identity/v3/role_assignments?user.id=88fee257b59d48f2b418ffa9e86b6f8e&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:06.485543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-e36920f4-f138-443d-aa2d-31df5d436904 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:06.486426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847a150-c9e8-4e3a-bbb7-ec3e051bbb30 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:06.494019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e36920f4-f138-443d-aa2d-31df5d436904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:06.571378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:06.572487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:06.585004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 92/183] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:06.585004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109bb6-8ca2-431f-95ee-3dc79a0214d9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.585004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109bb6-8ca2-431f-95ee-3dc79a0214d9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.585353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b109bb6-8ca2-431f-95ee-3dc79a0214d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.605908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-5b109bb6-8ca2-431f-95ee-3dc79a0214d9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:06.607113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5b109bb6-8ca2-431f-95ee-3dc79a0214d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:06.640000 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847a150-c9e8-4e3a-bbb7-ec3e051bbb30 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:06.676587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ce9c3-9699-4302-a589-c733e402d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:06.678138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ce9c3-9699-4302-a589-c733e402d463 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.678456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ce9c3-9699-4302-a589-c733e402d463 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.678733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ce9c3-9699-4302-a589-c733e402d463 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.679926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ce9c3-9699-4302-a589-c733e402d463 None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:06.681674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ce9c3-9699-4302-a589-c733e402d463 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:06.683579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 92/184] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:06 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:06.684770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90acb12-6105-4061-b1c1-eb049462fc0c None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.685063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90acb12-6105-4061-b1c1-eb049462fc0c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.686566 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90acb12-6105-4061-b1c1-eb049462fc0c None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.687545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 93/185] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:06.690816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90acb12-6105-4061-b1c1-eb049462fc0c None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:06.700735 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 93/186] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:06.701475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b109bb6-8ca2-431f-95ee-3dc79a0214d9 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:06.797038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90acb12-6105-4061-b1c1-eb049462fc0c None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:06.808799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b109bb6-8ca2-431f-95ee-3dc79a0214d9 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:06.821760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:06.830022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec2ed9-0ead-47ec-9bc1-be45391a055b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:06.830817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec2ed9-0ead-47ec-9bc1-be45391a055b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.833910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec2ed9-0ead-47ec-9bc1-be45391a055b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.833910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec2ed9-0ead-47ec-9bc1-be45391a055b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.834077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:06.842088 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec2ed9-0ead-47ec-9bc1-be45391a055b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:06.842429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b2fe95-7364-48db-a9d8-01503357bf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:06.843236 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fe95-7364-48db-a9d8-01503357bf4e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.843524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fe95-7364-48db-a9d8-01503357bf4e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.843801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b2fe95-7364-48db-a9d8-01503357bf4e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-93ec2ed9-0ead-47ec-9bc1-be45391a055b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:06.849818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:06.850716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 94/187] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:37:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:06.851890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec2ed9-0ead-47ec-9bc1-be45391a055b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:06.856509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b2fe95-7364-48db-a9d8-01503357bf4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-90b2fe95-7364-48db-a9d8-01503357bf4e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:06.864975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:06.866560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:06.867599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 94/188] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:37:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:06.872535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3807bb-19f0-4a59-bea5-c2f8c0e02ae5 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.872825 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3807bb-19f0-4a59-bea5-c2f8c0e02ae5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.873101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3807bb-19f0-4a59-bea5-c2f8c0e02ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.890467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-1a3807bb-19f0-4a59-bea5-c2f8c0e02ae5 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:06.891593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-1a3807bb-19f0-4a59-bea5-c2f8c0e02ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:06.950176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec2ed9-0ead-47ec-9bc1-be45391a055b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:06.956419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 95/189] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:06.958178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3807bb-19f0-4a59-bea5-c2f8c0e02ae5 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:06.983568 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c49d84-5b28-4b3a-bea8-9e29bb6550b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:06.984222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c49d84-5b28-4b3a-bea8-9e29bb6550b6 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:06.984507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c49d84-5b28-4b3a-bea8-9e29bb6550b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:06.984784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c49d84-5b28-4b3a-bea8-9e29bb6550b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:06.985637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c49d84-5b28-4b3a-bea8-9e29bb6550b6 None admin] RBAC: Adding query filter params (role.id=5f24daf2ba2b4f8fab95c87cefb52231, scope.project.id=0e628770474b423e99809dd568e34a0e, user.id=cf1ee67d28b6436fa3ec26eeae38390a) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:06.985950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c49d84-5b28-4b3a-bea8-9e29bb6550b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:06.987674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c49d84-5b28-4b3a-bea8-9e29bb6550b6 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:06.994055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 95/190] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:06 2022] GET /identity/v3/role_assignments?user.id=cf1ee67d28b6436fa3ec26eeae38390a&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=5f24daf2ba2b4f8fab95c87cefb52231 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:06.994849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c49d84-5b28-4b3a-bea8-9e29bb6550b6 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:07.056657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:07.072943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3807bb-19f0-4a59-bea5-c2f8c0e02ae5 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:07.089630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:07.112569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3363d-f28c-4950-ac57-5efa6a209fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:07.113062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c49d84-5b28-4b3a-bea8-9e29bb6550b6 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:07.113508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3363d-f28c-4950-ac57-5efa6a209fa2 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:07.113798 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3363d-f28c-4950-ac57-5efa6a209fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:07.114077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3363d-f28c-4950-ac57-5efa6a209fa2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:07.116943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3363d-f28c-4950-ac57-5efa6a209fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:07.118528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3363d-f28c-4950-ac57-5efa6a209fa2 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:07.125204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 96/191] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:07.125204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3363d-f28c-4950-ac57-5efa6a209fa2 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:07.151366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1182b-21b2-4149-9164-0b26e7ff921e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:07.151569 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1182b-21b2-4149-9164-0b26e7ff921e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:07.151773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1182b-21b2-4149-9164-0b26e7ff921e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:07.152053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1182b-21b2-4149-9164-0b26e7ff921e None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:07.152786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1182b-21b2-4149-9164-0b26e7ff921e None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:07.153090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1182b-21b2-4149-9164-0b26e7ff921e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:07.154513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1182b-21b2-4149-9164-0b26e7ff921e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:07.163932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 96/192] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:07.165274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1182b-21b2-4149-9164-0b26e7ff921e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:07.239253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3363d-f28c-4950-ac57-5efa6a209fa2 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:07.272940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a867543-2041-4cb4-bf50-a06f287741b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:07.273636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a867543-2041-4cb4-bf50-a06f287741b7 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:07.273924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a867543-2041-4cb4-bf50-a06f287741b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:07.274199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a867543-2041-4cb4-bf50-a06f287741b7 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:07.274410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1182b-21b2-4149-9164-0b26e7ff921e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:07.274940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a867543-2041-4cb4-bf50-a06f287741b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:07.275248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a867543-2041-4cb4-bf50-a06f287741b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:07.276640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a867543-2041-4cb4-bf50-a06f287741b7 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:07.284510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 97/193] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:07.285085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a867543-2041-4cb4-bf50-a06f287741b7 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:07.286789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:07.307907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99380fb4-64eb-470a-9ce6-434e8056ffc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:07.308694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99380fb4-64eb-470a-9ce6-434e8056ffc3 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:07.308980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99380fb4-64eb-470a-9ce6-434e8056ffc3 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:07.309277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99380fb4-64eb-470a-9ce6-434e8056ffc3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:07.319380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99380fb4-64eb-470a-9ce6-434e8056ffc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-99380fb4-64eb-470a-9ce6-434e8056ffc3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:07.327015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:07.327878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:07.328693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 97/194] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:07.329354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99380fb4-64eb-470a-9ce6-434e8056ffc3 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:07.335337 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:07.392065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a867543-2041-4cb4-bf50-a06f287741b7 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:07.425050 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a986dd4-6c44-4cbe-9f17-736534d6059f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:07.425688 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a986dd4-6c44-4cbe-9f17-736534d6059f None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:07.425834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99380fb4-64eb-470a-9ce6-434e8056ffc3 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:07.425979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a986dd4-6c44-4cbe-9f17-736534d6059f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:07.426246 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a986dd4-6c44-4cbe-9f17-736534d6059f None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:07.426944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a986dd4-6c44-4cbe-9f17-736534d6059f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:07.428420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a986dd4-6c44-4cbe-9f17-736534d6059f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:07.458075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b19b86-f9fb-47a2-bf63-bd84b4806784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:07.459488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19b86-f9fb-47a2-bf63-bd84b4806784 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:07.459760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19b86-f9fb-47a2-bf63-bd84b4806784 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:07.460040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b19b86-f9fb-47a2-bf63-bd84b4806784 None admin] PATH_INFO: `/v3/users/c9ae878d0d1841e79280707ee29a1b60` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:07.472851 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b19b86-f9fb-47a2-bf63-bd84b4806784 None admin] RBAC: Authorizing `identity:get_user(user_id=c9ae878d0d1841e79280707ee29a1b60)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:07.474381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b19b86-f9fb-47a2-bf63-bd84b4806784 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:07.476586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 98/195] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:07 2022] GET /identity/v3/users/c9ae878d0d1841e79280707ee29a1b60 => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:07.477368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b19b86-f9fb-47a2-bf63-bd84b4806784 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:07.505632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:07.522064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 98/196] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:37:07 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:07.522064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a986dd4-6c44-4cbe-9f17-736534d6059f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:07.592948 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b19b86-f9fb-47a2-bf63-bd84b4806784 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:07.629940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:07.630470 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f991c93-1462-4071-84bc-ecb79ea304b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:07.631611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f991c93-1462-4071-84bc-ecb79ea304b8 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:07.631722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f991c93-1462-4071-84bc-ecb79ea304b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:07.631955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f991c93-1462-4071-84bc-ecb79ea304b8 None admin] PATH_INFO: `/v3/users/c9ae878d0d1841e79280707ee29a1b60` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:07.633071 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a986dd4-6c44-4cbe-9f17-736534d6059f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:07.633559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f991c93-1462-4071-84bc-ecb79ea304b8 None admin] RBAC: Authorizing `identity:get_user(user_id=c9ae878d0d1841e79280707ee29a1b60)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:07.635050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f991c93-1462-4071-84bc-ecb79ea304b8 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:07.636866 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 99/197] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:07 2022] GET /identity/v3/users/c9ae878d0d1841e79280707ee29a1b60 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:07.637630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f991c93-1462-4071-84bc-ecb79ea304b8 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:07.676446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d348c-4e21-4a66-bc86-ddd3df49311b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:07.677282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d348c-4e21-4a66-bc86-ddd3df49311b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:07.677406 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d348c-4e21-4a66-bc86-ddd3df49311b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:07.677654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831d348c-4e21-4a66-bc86-ddd3df49311b None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:07.678535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d348c-4e21-4a66-bc86-ddd3df49311b None admin] RBAC: Adding query filter params (name=reader) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:07.678713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d348c-4e21-4a66-bc86-ddd3df49311b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:07.680103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831d348c-4e21-4a66-bc86-ddd3df49311b None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:07.690706 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 99/198] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:07.691338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d348c-4e21-4a66-bc86-ddd3df49311b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:07.761212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:07.766589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f991c93-1462-4071-84bc-ecb79ea304b8 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:07.808138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831d348c-4e21-4a66-bc86-ddd3df49311b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:07.810658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1c906-bcef-4971-8ee1-a6bc5570d74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:07.811332 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1c906-bcef-4971-8ee1-a6bc5570d74d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:07.811621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1c906-bcef-4971-8ee1-a6bc5570d74d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:07.811902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1c906-bcef-4971-8ee1-a6bc5570d74d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:07.849744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94985c94-6ec9-4c15-b6a7-40d5b3ac7229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:07.850283 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94985c94-6ec9-4c15-b6a7-40d5b3ac7229 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:07.850372 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94985c94-6ec9-4c15-b6a7-40d5b3ac7229 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:07.850630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94985c94-6ec9-4c15-b6a7-40d5b3ac7229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:07.890476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1c906-bcef-4971-8ee1-a6bc5570d74d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:07.892381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1c906-bcef-4971-8ee1-a6bc5570d74d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:07.895123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:07.941436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94985c94-6ec9-4c15-b6a7-40d5b3ac7229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:07.942831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94985c94-6ec9-4c15-b6a7-40d5b3ac7229 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:08.013992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:08.021473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 100/199] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:08.023027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1c906-bcef-4971-8ee1-a6bc5570d74d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:08.055640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 100/200] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:08.056485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94985c94-6ec9-4c15-b6a7-40d5b3ac7229 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:08.134092 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:08.172118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1c906-bcef-4971-8ee1-a6bc5570d74d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:08.178893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94985c94-6ec9-4c15-b6a7-40d5b3ac7229 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:08.211583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb624fd8-37c2-4cae-90b5-dc93c6c8c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:08.212375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb624fd8-37c2-4cae-90b5-dc93c6c8c00a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:08.212658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb624fd8-37c2-4cae-90b5-dc93c6c8c00a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:08.213034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb624fd8-37c2-4cae-90b5-dc93c6c8c00a None admin] PATH_INFO: `/v3/users/3628f92de2bf465d90beaeb1f935f0c7` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:08.219814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb624fd8-37c2-4cae-90b5-dc93c6c8c00a None admin] RBAC: Authorizing `identity:get_user(user_id=3628f92de2bf465d90beaeb1f935f0c7)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:08.219814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb624fd8-37c2-4cae-90b5-dc93c6c8c00a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:08.219814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 101/201] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:08 2022] GET /identity/v3/users/3628f92de2bf465d90beaeb1f935f0c7 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:08.219814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb624fd8-37c2-4cae-90b5-dc93c6c8c00a None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:08.223578 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a0a52-2771-4c6f-bb20-f931308f3af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:08.224474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a0a52-2771-4c6f-bb20-f931308f3af1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:08.224791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a0a52-2771-4c6f-bb20-f931308f3af1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:08.225072 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a0a52-2771-4c6f-bb20-f931308f3af1 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:08.226381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a0a52-2771-4c6f-bb20-f931308f3af1 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:08.227859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a0a52-2771-4c6f-bb20-f931308f3af1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:08.230038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 101/202] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:08 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:08.230867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a0a52-2771-4c6f-bb20-f931308f3af1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:08.289458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:08.328765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb624fd8-37c2-4cae-90b5-dc93c6c8c00a None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:08.342670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a0a52-2771-4c6f-bb20-f931308f3af1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:08.373013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2373bb-0ff1-4af4-b664-64f6f66a9d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:08.374117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2373bb-0ff1-4af4-b664-64f6f66a9d31 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:08.374521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2373bb-0ff1-4af4-b664-64f6f66a9d31 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:08.374914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2373bb-0ff1-4af4-b664-64f6f66a9d31 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:08.376298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2373bb-0ff1-4af4-b664-64f6f66a9d31 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:08.378003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2373bb-0ff1-4af4-b664-64f6f66a9d31 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:08.379237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:08.382020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 102/203] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:08 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:08.383333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2373bb-0ff1-4af4-b664-64f6f66a9d31 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:08.385552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7bc7a-5772-4f2c-9655-b948346e8a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:08.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7bc7a-5772-4f2c-9655-b948346e8a1f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:08.387070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7bc7a-5772-4f2c-9655-b948346e8a1f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:08.387352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c7bc7a-5772-4f2c-9655-b948346e8a1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:08.478862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7bc7a-5772-4f2c-9655-b948346e8a1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:08.480843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c7bc7a-5772-4f2c-9655-b948346e8a1f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:08.494088 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2373bb-0ff1-4af4-b664-64f6f66a9d31 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:08.532949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c151b87-299d-4cbd-b77b-55fa78de1422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:08.534034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c151b87-299d-4cbd-b77b-55fa78de1422 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:08.534523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c151b87-299d-4cbd-b77b-55fa78de1422 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:08.535020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c151b87-299d-4cbd-b77b-55fa78de1422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:08.536076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c151b87-299d-4cbd-b77b-55fa78de1422 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f, user.id=c9ae878d0d1841e79280707ee29a1b60) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:08.536651 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c151b87-299d-4cbd-b77b-55fa78de1422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:08.539036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c151b87-299d-4cbd-b77b-55fa78de1422 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:08.541548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:08.546479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 103/204] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:08 2022] GET /identity/v3/role_assignments?user.id=c9ae878d0d1841e79280707ee29a1b60&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:08.547752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c151b87-299d-4cbd-b77b-55fa78de1422 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:08.606878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 102/205] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:08.618400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7bc7a-5772-4f2c-9655-b948346e8a1f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:08.677059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c151b87-299d-4cbd-b77b-55fa78de1422 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:08.681455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:08.717745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb792a9-dc17-4c06-b1df-e461178e9b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:08.718443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb792a9-dc17-4c06-b1df-e461178e9b37 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:08.719953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb792a9-dc17-4c06-b1df-e461178e9b37 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:08.719953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb792a9-dc17-4c06-b1df-e461178e9b37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:08.720134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb792a9-dc17-4c06-b1df-e461178e9b37 None admin] RBAC: Adding query filter params (role.id=5f24daf2ba2b4f8fab95c87cefb52231, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f, user.id=c9ae878d0d1841e79280707ee29a1b60) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:08.720464 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb792a9-dc17-4c06-b1df-e461178e9b37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:08.721938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb792a9-dc17-4c06-b1df-e461178e9b37 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:08.728947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 104/206] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:08 2022] GET /identity/v3/role_assignments?user.id=c9ae878d0d1841e79280707ee29a1b60&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=5f24daf2ba2b4f8fab95c87cefb52231 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:08.812009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c7bc7a-5772-4f2c-9655-b948346e8a1f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:08.864038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179b11a-e070-4258-a954-4a7017aa762a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:08.865766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179b11a-e070-4258-a954-4a7017aa762a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:08.866222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179b11a-e070-4258-a954-4a7017aa762a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:08.866644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179b11a-e070-4258-a954-4a7017aa762a None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:08.868223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179b11a-e070-4258-a954-4a7017aa762a None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:08.869941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179b11a-e070-4258-a954-4a7017aa762a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:08.872404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 103/207] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:08 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:08.878384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb792a9-dc17-4c06-b1df-e461178e9b37 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:08.955620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:08.995430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb792a9-dc17-4c06-b1df-e461178e9b37 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:09.031611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb93ae1-15cc-4623-a0ac-938440fbda87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:09.032279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb93ae1-15cc-4623-a0ac-938440fbda87 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:09.032568 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb93ae1-15cc-4623-a0ac-938440fbda87 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:09.032848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb93ae1-15cc-4623-a0ac-938440fbda87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:09.033711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb93ae1-15cc-4623-a0ac-938440fbda87 None admin] RBAC: Adding query filter params (role.id=43a16638caac4dd8acae2987c010b48a, scope.project.id=0e628770474b423e99809dd568e34a0e, user.id=3628f92de2bf465d90beaeb1f935f0c7) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:09.034029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb93ae1-15cc-4623-a0ac-938440fbda87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:09.035477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb93ae1-15cc-4623-a0ac-938440fbda87 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:09.041916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 105/208] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:08 2022] GET /identity/v3/role_assignments?user.id=3628f92de2bf465d90beaeb1f935f0c7&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=43a16638caac4dd8acae2987c010b48a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:09.992386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefe026-820f-4976-adc0-07be8b29689e None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:09.992907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefe026-820f-4976-adc0-07be8b29689e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:09.992907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefe026-820f-4976-adc0-07be8b29689e None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:09.994496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 104/209] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:10.000270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b09dfa-ca92-432e-85a9-791e7ea8a036 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:10.000742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b09dfa-ca92-432e-85a9-791e7ea8a036 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:10.000816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b09dfa-ca92-432e-85a9-791e7ea8a036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:10.018604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f8b09dfa-ca92-432e-85a9-791e7ea8a036 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:10.019799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f8b09dfa-ca92-432e-85a9-791e7ea8a036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:10.043256 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fb28d-1f5d-4995-bc02-def4e9e2cb48 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:10.043549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fb28d-1f5d-4995-bc02-def4e9e2cb48 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:10.043838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fb28d-1f5d-4995-bc02-def4e9e2cb48 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:10.045209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 105/210] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:10.050535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bd786-6215-4308-919b-f64b3baa63ce None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:10.050824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bd786-6215-4308-919b-f64b3baa63ce None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:10.051109 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bd786-6215-4308-919b-f64b3baa63ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:10.072350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-876bd786-6215-4308-919b-f64b3baa63ce None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:10.073957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-876bd786-6215-4308-919b-f64b3baa63ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:10.100301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 106/211] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:10.124970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b09dfa-ca92-432e-85a9-791e7ea8a036 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:10.155028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 106/212] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:10.178728 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bd786-6215-4308-919b-f64b3baa63ce None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:10.185358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:10.207185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:10.229853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b09dfa-ca92-432e-85a9-791e7ea8a036 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:10.264390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0832f-9c17-49e7-b835-9e5053c70f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:10.265052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0832f-9c17-49e7-b835-9e5053c70f47 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:10.265370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0832f-9c17-49e7-b835-9e5053c70f47 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:10.265678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0832f-9c17-49e7-b835-9e5053c70f47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:10.266876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0832f-9c17-49e7-b835-9e5053c70f47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:10.268368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b0832f-9c17-49e7-b835-9e5053c70f47 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:10.270324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 107/213] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:10.276228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0832f-9c17-49e7-b835-9e5053c70f47 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:10.282620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bd786-6215-4308-919b-f64b3baa63ce None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:10.317970 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83941abf-e09f-4ef6-9b91-55bd4db126bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:10.319546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83941abf-e09f-4ef6-9b91-55bd4db126bb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:10.320008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83941abf-e09f-4ef6-9b91-55bd4db126bb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:10.320460 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83941abf-e09f-4ef6-9b91-55bd4db126bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:10.331444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83941abf-e09f-4ef6-9b91-55bd4db126bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-83941abf-e09f-4ef6-9b91-55bd4db126bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:10.340030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:10.340880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:10.341708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:10.349053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 107/214] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:10.350250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83941abf-e09f-4ef6-9b91-55bd4db126bb None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:10.378355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0832f-9c17-49e7-b835-9e5053c70f47 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:10.405683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:10.416543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fb4aa-8984-4ce9-b3aa-8a4e74ae8747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:10.417494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fb4aa-8984-4ce9-b3aa-8a4e74ae8747 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:10.417886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fb4aa-8984-4ce9-b3aa-8a4e74ae8747 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:10.418268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fb4aa-8984-4ce9-b3aa-8a4e74ae8747 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:10.419081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fb4aa-8984-4ce9-b3aa-8a4e74ae8747 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:10.421105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fb4aa-8984-4ce9-b3aa-8a4e74ae8747 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:10.445940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:10.457686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83941abf-e09f-4ef6-9b91-55bd4db126bb None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:10.491272 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15d780-f9b1-48da-9c86-1fb8928aa3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:10.491990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15d780-f9b1-48da-9c86-1fb8928aa3fb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:10.492282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15d780-f9b1-48da-9c86-1fb8928aa3fb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:10.492555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15d780-f9b1-48da-9c86-1fb8928aa3fb None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:10.493322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15d780-f9b1-48da-9c86-1fb8928aa3fb None admin] RBAC: Adding query filter params (name=member) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:10.493641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15d780-f9b1-48da-9c86-1fb8928aa3fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:10.495043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff15d780-f9b1-48da-9c86-1fb8928aa3fb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:10.503488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 108/215] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:10.509403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15d780-f9b1-48da-9c86-1fb8928aa3fb None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:10.526037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 108/216] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:37:10 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:37:10.636375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15d780-f9b1-48da-9c86-1fb8928aa3fb None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:10.674139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfdba76-7adf-4c85-96fc-04b11369dd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:10.675045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdba76-7adf-4c85-96fc-04b11369dd1f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:10.675331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdba76-7adf-4c85-96fc-04b11369dd1f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:10.675605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfdba76-7adf-4c85-96fc-04b11369dd1f None admin] PATH_INFO: `/v3/users/83bfb2eb1a1f4b4eb9d792f87d3ed25c` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:10.692049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdba76-7adf-4c85-96fc-04b11369dd1f None admin] RBAC: Authorizing `identity:get_user(user_id=83bfb2eb1a1f4b4eb9d792f87d3ed25c)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:10.693593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfdba76-7adf-4c85-96fc-04b11369dd1f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:10.696221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 109/217] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:10 2022] GET /identity/v3/users/83bfb2eb1a1f4b4eb9d792f87d3ed25c => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:10.702485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fb4aa-8984-4ce9-b3aa-8a4e74ae8747 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:10.836991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fb4aa-8984-4ce9-b3aa-8a4e74ae8747 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:10.874216 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46d393-7b54-4e61-b80d-cd2108c8192e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:10.874883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46d393-7b54-4e61-b80d-cd2108c8192e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:10.875169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46d393-7b54-4e61-b80d-cd2108c8192e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:10.875441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46d393-7b54-4e61-b80d-cd2108c8192e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:10.925961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:10.946862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46d393-7b54-4e61-b80d-cd2108c8192e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:10.948361 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e46d393-7b54-4e61-b80d-cd2108c8192e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:11.040506 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816c7e-07c8-4eec-a000-f42ca63807b8 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:11.041008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816c7e-07c8-4eec-a000-f42ca63807b8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:11.041082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75816c7e-07c8-4eec-a000-f42ca63807b8 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:11.042597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 110/218] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:37:11.047757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaa5d7-d48a-41e8-91f4-11365da9bb96 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:11.048048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaa5d7-d48a-41e8-91f4-11365da9bb96 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:11.048330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaa5d7-d48a-41e8-91f4-11365da9bb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:11.067632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6edaa5d7-d48a-41e8-91f4-11365da9bb96 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:11.069084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6edaa5d7-d48a-41e8-91f4-11365da9bb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:11.071159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:11.085814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 109/219] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:11.087869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a8b6-c682-4ffe-ac0a-902858093559 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:11.088106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a8b6-c682-4ffe-ac0a-902858093559 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:11.088418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a8b6-c682-4ffe-ac0a-902858093559 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:11.090001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 110/220] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:11.093068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246a8b6-c682-4ffe-ac0a-902858093559 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:11.155957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 111/221] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:11.157278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184098c9-6d9a-4a91-9e14-b4b9749bb00c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:11.157576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184098c9-6d9a-4a91-9e14-b4b9749bb00c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:11.157856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184098c9-6d9a-4a91-9e14-b4b9749bb00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:11.176975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-184098c9-6d9a-4a91-9e14-b4b9749bb00c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:11.178975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-184098c9-6d9a-4a91-9e14-b4b9749bb00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:11.210923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:11.231888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246a8b6-c682-4ffe-ac0a-902858093559 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:11.269646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 112/222] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:11.270675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184098c9-6d9a-4a91-9e14-b4b9749bb00c None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:11.270898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b145c-2098-447b-b796-e675ea62c963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:11.271909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b145c-2098-447b-b796-e675ea62c963 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:11.272289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b145c-2098-447b-b796-e675ea62c963 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:11.272692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539b145c-2098-447b-b796-e675ea62c963 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:11.274118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b145c-2098-447b-b796-e675ea62c963 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:11.275723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539b145c-2098-447b-b796-e675ea62c963 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:11.278003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 111/223] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:11 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:11.284252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b145c-2098-447b-b796-e675ea62c963 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:11.367739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:11.401971 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184098c9-6d9a-4a91-9e14-b4b9749bb00c None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:11.408990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539b145c-2098-447b-b796-e675ea62c963 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:11.440268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cced6b8-b673-43ea-b0e0-ee33e4b91f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:11.441693 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cced6b8-b673-43ea-b0e0-ee33e4b91f22 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:11.442415 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cced6b8-b673-43ea-b0e0-ee33e4b91f22 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:11.443329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cced6b8-b673-43ea-b0e0-ee33e4b91f22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:11.454563 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca7695-6022-48f1-bd99-4195fce77074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:11.455231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca7695-6022-48f1-bd99-4195fce77074 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:11.455514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca7695-6022-48f1-bd99-4195fce77074 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:11.455791 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca7695-6022-48f1-bd99-4195fce77074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:11.456622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca7695-6022-48f1-bd99-4195fce77074 None admin] RBAC: Adding query filter params (role.id=0960434fd9da404ea96841eb4306eaae, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f, user.id=83bfb2eb1a1f4b4eb9d792f87d3ed25c) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:11.456938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca7695-6022-48f1-bd99-4195fce77074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:11.458100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cced6b8-b673-43ea-b0e0-ee33e4b91f22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:11.458536 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca7695-6022-48f1-bd99-4195fce77074 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-2cced6b8-b673-43ea-b0e0-ee33e4b91f22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:11.467209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:11.468097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:11.468877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:11.471391 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 113/225] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:37:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:11.471828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 112/225] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:11 2022] GET /identity/v3/role_assignments?user.id=83bfb2eb1a1f4b4eb9d792f87d3ed25c&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=0960434fd9da404ea96841eb4306eaae => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:11.472759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cced6b8-b673-43ea-b0e0-ee33e4b91f22 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:11.478100 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca7695-6022-48f1-bd99-4195fce77074 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:11.504355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:11.599905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cced6b8-b673-43ea-b0e0-ee33e4b91f22 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:11.633843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:11.648613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc3a11-0c05-4a13-9f27-f7db36081fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:11.648928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc3a11-0c05-4a13-9f27-f7db36081fad None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:11.650985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc3a11-0c05-4a13-9f27-f7db36081fad None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:11.650985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc3a11-0c05-4a13-9f27-f7db36081fad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:11.676703 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc3a11-0c05-4a13-9f27-f7db36081fad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-d0bc3a11-0c05-4a13-9f27-f7db36081fad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:11.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:11.699258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 114/226] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:37:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:11.700036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc3a11-0c05-4a13-9f27-f7db36081fad None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:11.703234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca7695-6022-48f1-bd99-4195fce77074 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:11.756989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab09772-2622-48e6-96bb-cc4a1bb978a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:11.758106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab09772-2622-48e6-96bb-cc4a1bb978a7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:11.758404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab09772-2622-48e6-96bb-cc4a1bb978a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:11.758688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab09772-2622-48e6-96bb-cc4a1bb978a7 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:11.759712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab09772-2622-48e6-96bb-cc4a1bb978a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:11.760030 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab09772-2622-48e6-96bb-cc4a1bb978a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:11.761560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab09772-2622-48e6-96bb-cc4a1bb978a7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:11.773527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 113/227] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:11.779377 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab09772-2622-48e6-96bb-cc4a1bb978a7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:11.882186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc3a11-0c05-4a13-9f27-f7db36081fad None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:11.931643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:11.957041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ec5dd-3063-4f59-b703-81fe31eed9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:11.958074 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ec5dd-3063-4f59-b703-81fe31eed9df None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:11.960459 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ec5dd-3063-4f59-b703-81fe31eed9df None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:11.960938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ec5dd-3063-4f59-b703-81fe31eed9df None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:11.961908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ec5dd-3063-4f59-b703-81fe31eed9df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:11.963777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ec5dd-3063-4f59-b703-81fe31eed9df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:11.965405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ec5dd-3063-4f59-b703-81fe31eed9df None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:11.984697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 115/228] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:11.985486 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab09772-2622-48e6-96bb-cc4a1bb978a7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:11.992382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ec5dd-3063-4f59-b703-81fe31eed9df None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:12.032576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d641a83-ec93-42b7-b301-e446c3335a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:12.034899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d641a83-ec93-42b7-b301-e446c3335a32 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:12.036232 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d641a83-ec93-42b7-b301-e446c3335a32 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:12.038247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d641a83-ec93-42b7-b301-e446c3335a32 None admin] PATH_INFO: `/v3/users/c9ae878d0d1841e79280707ee29a1b60` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:12.039760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d641a83-ec93-42b7-b301-e446c3335a32 None admin] RBAC: Authorizing `identity:get_user(user_id=c9ae878d0d1841e79280707ee29a1b60)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:12.042520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d641a83-ec93-42b7-b301-e446c3335a32 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:12.049003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 114/229] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:11 2022] GET /identity/v3/users/c9ae878d0d1841e79280707ee29a1b60 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:12.053817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d641a83-ec93-42b7-b301-e446c3335a32 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:12.098472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:12.175983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ec5dd-3063-4f59-b703-81fe31eed9df None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:12.215837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7af32-73de-4909-902c-b284d63e7768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:12.217087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7af32-73de-4909-902c-b284d63e7768 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:12.218635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7af32-73de-4909-902c-b284d63e7768 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:12.219609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7af32-73de-4909-902c-b284d63e7768 None admin] PATH_INFO: `/v3/users/c9ae878d0d1841e79280707ee29a1b60` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:12.221906 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7af32-73de-4909-902c-b284d63e7768 None admin] RBAC: Authorizing `identity:get_user(user_id=c9ae878d0d1841e79280707ee29a1b60)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:12.223832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d641a83-ec93-42b7-b301-e446c3335a32 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:12.226345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7af32-73de-4909-902c-b284d63e7768 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:12.229505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 116/230] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:11 2022] GET /identity/v3/users/c9ae878d0d1841e79280707ee29a1b60 => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:12.235912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7af32-73de-4909-902c-b284d63e7768 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:12.294994 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998e181-338c-4c84-b307-6d3adcf9e656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:12.297127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998e181-338c-4c84-b307-6d3adcf9e656 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:12.297714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998e181-338c-4c84-b307-6d3adcf9e656 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:12.298237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998e181-338c-4c84-b307-6d3adcf9e656 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:12.299645 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998e181-338c-4c84-b307-6d3adcf9e656 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:12.301591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998e181-338c-4c84-b307-6d3adcf9e656 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:12.309151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 115/231] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:12 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:12.310562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998e181-338c-4c84-b307-6d3adcf9e656 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:12.336640 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:12.447386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7af32-73de-4909-902c-b284d63e7768 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:12.457904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998e181-338c-4c84-b307-6d3adcf9e656 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:12.481552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:12.513794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0c940-0367-4037-bbc3-1d61883d87e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:12.516736 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0c940-0367-4037-bbc3-1d61883d87e2 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:12.517046 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0c940-0367-4037-bbc3-1d61883d87e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:12.517355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0c940-0367-4037-bbc3-1d61883d87e2 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:12.519165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0c940-0367-4037-bbc3-1d61883d87e2 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:12.521029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d0c940-0367-4037-bbc3-1d61883d87e2 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:12.524897 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 117/232] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:12 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:12.526232 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f39f1b-5126-4ebe-88be-ac59b4163f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:12.527285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f39f1b-5126-4ebe-88be-ac59b4163f2f None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:12.527712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f39f1b-5126-4ebe-88be-ac59b4163f2f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:12.528268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f39f1b-5126-4ebe-88be-ac59b4163f2f None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/users/c9ae878d0d1841e79280707ee29a1b60/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:12.530019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0c940-0367-4037-bbc3-1d61883d87e2 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:12.531947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f39f1b-5126-4ebe-88be-ac59b4163f2f None admin] RBAC: Authorizing `identity:create_grant(project_id=4088a75edb3f4c128c5f08a69f835d4f, user_id=c9ae878d0d1841e79280707ee29a1b60, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:12.535234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f39f1b-5126-4ebe-88be-ac59b4163f2f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:12.624521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 116/233] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:37:12 2022] PUT /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/users/c9ae878d0d1841e79280707ee29a1b60/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:37:12.661610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0c940-0367-4037-bbc3-1d61883d87e2 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:12.700077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cb4e6-2bfe-45fb-b83e-7de84659300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:12.701098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cb4e6-2bfe-45fb-b83e-7de84659300f None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:12.701420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cb4e6-2bfe-45fb-b83e-7de84659300f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:12.701785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cb4e6-2bfe-45fb-b83e-7de84659300f None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/users/c9ae878d0d1841e79280707ee29a1b60/roles/5f24daf2ba2b4f8fab95c87cefb52231` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:12.704268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cb4e6-2bfe-45fb-b83e-7de84659300f None admin] RBAC: Authorizing `identity:create_grant(project_id=4088a75edb3f4c128c5f08a69f835d4f, user_id=c9ae878d0d1841e79280707ee29a1b60, role_id=5f24daf2ba2b4f8fab95c87cefb52231)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:12.705798 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cb4e6-2bfe-45fb-b83e-7de84659300f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:12.825769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 118/234] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:37:12 2022] PUT /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/users/c9ae878d0d1841e79280707ee29a1b60/roles/5f24daf2ba2b4f8fab95c87cefb52231 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:37:13.106101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f0a74-9d9c-4bde-8cb5-1ddf434f2cff None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:13.106756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f0a74-9d9c-4bde-8cb5-1ddf434f2cff None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:13.107259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f0a74-9d9c-4bde-8cb5-1ddf434f2cff None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:13.109059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 117/235] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:13.115129 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891852c-2320-470b-87e0-7e9cb7721120 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:13.116043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891852c-2320-470b-87e0-7e9cb7721120 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:13.116521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891852c-2320-470b-87e0-7e9cb7721120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:13.136844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-c891852c-2320-470b-87e0-7e9cb7721120 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:13.138071 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c891852c-2320-470b-87e0-7e9cb7721120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:13.188024 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:13.225697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 119/236] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:13.244319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5bc97-20bc-4301-99a6-5bfa3495b1cd None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:13.245266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5bc97-20bc-4301-99a6-5bfa3495b1cd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:13.245773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5bc97-20bc-4301-99a6-5bfa3495b1cd None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:13.247626 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 118/237] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:13.252565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6b7cf-49dc-455a-a4b8-c16c5c3ac5eb None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:13.252851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6b7cf-49dc-455a-a4b8-c16c5c3ac5eb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:13.253123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6b7cf-49dc-455a-a4b8-c16c5c3ac5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:13.258802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5bc97-20bc-4301-99a6-5bfa3495b1cd None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:13.273976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-bae6b7cf-49dc-455a-a4b8-c16c5c3ac5eb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:13.275521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-bae6b7cf-49dc-455a-a4b8-c16c5c3ac5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:13.367444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 120/238] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:13.391952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6b7cf-49dc-455a-a4b8-c16c5c3ac5eb None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:13.431376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:13.505653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6b7cf-49dc-455a-a4b8-c16c5c3ac5eb None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:13.515641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:13.521908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5bc97-20bc-4301-99a6-5bfa3495b1cd None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:13.546138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e58d8-54f5-4fab-9bb7-e7797e94396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:13.547373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e58d8-54f5-4fab-9bb7-e7797e94396f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:13.547803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e58d8-54f5-4fab-9bb7-e7797e94396f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:13.548263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e58d8-54f5-4fab-9bb7-e7797e94396f None admin] PATH_INFO: `/v3/domains/default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:13.549600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e58d8-54f5-4fab-9bb7-e7797e94396f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:13.551204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e58d8-54f5-4fab-9bb7-e7797e94396f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:13.553890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 121/239] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:13.559934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e58d8-54f5-4fab-9bb7-e7797e94396f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:13.602510 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefcbcf-a705-48ae-916b-c63dbefd0325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:13.603309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefcbcf-a705-48ae-916b-c63dbefd0325 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:13.603590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefcbcf-a705-48ae-916b-c63dbefd0325 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:13.603860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fefcbcf-a705-48ae-916b-c63dbefd0325 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:13.626281 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fefcbcf-a705-48ae-916b-c63dbefd0325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-3fefcbcf-a705-48ae-916b-c63dbefd0325 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:13.642086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:13.643051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:13.644450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 119/240] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:13.649553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefcbcf-a705-48ae-916b-c63dbefd0325 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:13.683017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e58d8-54f5-4fab-9bb7-e7797e94396f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:13.758583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ea6f3-c6a3-4dd9-ba2b-8a706806e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:13.759273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ea6f3-c6a3-4dd9-ba2b-8a706806e748 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:13.759560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ea6f3-c6a3-4dd9-ba2b-8a706806e748 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:13.759846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ea6f3-c6a3-4dd9-ba2b-8a706806e748 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:13.760551 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ea6f3-c6a3-4dd9-ba2b-8a706806e748 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:13.762027 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ea6f3-c6a3-4dd9-ba2b-8a706806e748 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:13.770156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fefcbcf-a705-48ae-916b-c63dbefd0325 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:13.801199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:13.805910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eacba7-ce7a-467b-a190-95a87adbe320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:13.805910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eacba7-ce7a-467b-a190-95a87adbe320 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:13.805910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eacba7-ce7a-467b-a190-95a87adbe320 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:13.806137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eacba7-ce7a-467b-a190-95a87adbe320 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:13.806827 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eacba7-ce7a-467b-a190-95a87adbe320 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:13.807130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eacba7-ce7a-467b-a190-95a87adbe320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:13.808508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eacba7-ce7a-467b-a190-95a87adbe320 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:13.817164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 120/241] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:13.822768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eacba7-ce7a-467b-a190-95a87adbe320 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:13.885302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 122/242] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:37:13 2022] POST /identity/v3/users => generated 293 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:13.931238 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eacba7-ce7a-467b-a190-95a87adbe320 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:13.967737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa05764-63d2-48aa-aaae-3503fc275137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:13.968622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa05764-63d2-48aa-aaae-3503fc275137 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:13.968901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa05764-63d2-48aa-aaae-3503fc275137 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:13.969170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa05764-63d2-48aa-aaae-3503fc275137 None admin] PATH_INFO: `/v3/users/f32656e3430d43b4af5f4f34238f26ac` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:13.984067 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa05764-63d2-48aa-aaae-3503fc275137 None admin] RBAC: Authorizing `identity:get_user(user_id=f32656e3430d43b4af5f4f34238f26ac)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:13.985603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa05764-63d2-48aa-aaae-3503fc275137 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:13.987999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 121/243] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:13 2022] GET /identity/v3/users/f32656e3430d43b4af5f4f34238f26ac => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:13.993342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ea6f3-c6a3-4dd9-ba2b-8a706806e748 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:14.053943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:14.142263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ea6f3-c6a3-4dd9-ba2b-8a706806e748 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:14.165850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cc3f2-39a9-42cf-8cce-d155d48bad59 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:14.166503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cc3f2-39a9-42cf-8cce-d155d48bad59 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:14.166971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cc3f2-39a9-42cf-8cce-d155d48bad59 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:14.168805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 122/244] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:14.179177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a4410-63f8-4fcd-9d2a-2fb1489b0a4f None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:14.179781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a4410-63f8-4fcd-9d2a-2fb1489b0a4f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:14.180060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a4410-63f8-4fcd-9d2a-2fb1489b0a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:14.181000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89db75-d7ca-4a16-8f7f-e776d3b948c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:14.181771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89db75-d7ca-4a16-8f7f-e776d3b948c8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:14.182056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89db75-d7ca-4a16-8f7f-e776d3b948c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:14.182336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89db75-d7ca-4a16-8f7f-e776d3b948c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:14.205185 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b79a4410-63f8-4fcd-9d2a-2fb1489b0a4f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:14.206413 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b79a4410-63f8-4fcd-9d2a-2fb1489b0a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:14.266309 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89db75-d7ca-4a16-8f7f-e776d3b948c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:14.268540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d89db75-d7ca-4a16-8f7f-e776d3b948c8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:14.273716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:14.303885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 123/245] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:14.309599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:14.328360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a4410-63f8-4fcd-9d2a-2fb1489b0a4f None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:14.411199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 123/246] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:14.417987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89db75-d7ca-4a16-8f7f-e776d3b948c8 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:14.475569 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a4410-63f8-4fcd-9d2a-2fb1489b0a4f None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:14.522140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b9c66-4503-41b4-8bad-de7833857017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:14.523487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b9c66-4503-41b4-8bad-de7833857017 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:14.523777 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b9c66-4503-41b4-8bad-de7833857017 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:14.524056 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b9c66-4503-41b4-8bad-de7833857017 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:14.537344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b9c66-4503-41b4-8bad-de7833857017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:14.549157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89db75-d7ca-4a16-8f7f-e776d3b948c8 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-e78b9c66-4503-41b4-8bad-de7833857017 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:14.549330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:14.550252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:14.551218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 124/247] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:14.558725 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b9c66-4503-41b4-8bad-de7833857017 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:14.590275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:14.598207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3f685-51e2-4643-815b-ae8d914eefe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:14.599075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3f685-51e2-4643-815b-ae8d914eefe7 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:14.599355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3f685-51e2-4643-815b-ae8d914eefe7 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:14.599633 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3f685-51e2-4643-815b-ae8d914eefe7 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:14.601470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3f685-51e2-4643-815b-ae8d914eefe7 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:14.603333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3f685-51e2-4643-815b-ae8d914eefe7 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:14.604715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 124/248] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:14 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:14.610641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3f685-51e2-4643-815b-ae8d914eefe7 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:14.622879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:14.693559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b9c66-4503-41b4-8bad-de7833857017 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:14.735399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd37d5-5d0c-4d9b-8250-5e15cb10cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:14.740022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd37d5-5d0c-4d9b-8250-5e15cb10cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:14.740022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd37d5-5d0c-4d9b-8250-5e15cb10cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:14.740022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd37d5-5d0c-4d9b-8250-5e15cb10cfe7 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:14.740022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd37d5-5d0c-4d9b-8250-5e15cb10cfe7 None admin] RBAC: Adding query filter params (name=member) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:14.740022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd37d5-5d0c-4d9b-8250-5e15cb10cfe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:14.740022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd37d5-5d0c-4d9b-8250-5e15cb10cfe7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:14.749800 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 125/249] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:14.751938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3f685-51e2-4643-815b-ae8d914eefe7 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:14.755487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd37d5-5d0c-4d9b-8250-5e15cb10cfe7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:14.797455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20870389-aac4-460a-8450-9f5189ab27aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:14.798358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20870389-aac4-460a-8450-9f5189ab27aa None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:14.798781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20870389-aac4-460a-8450-9f5189ab27aa None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:14.799222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20870389-aac4-460a-8450-9f5189ab27aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:14.800672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20870389-aac4-460a-8450-9f5189ab27aa None admin] RBAC: Adding query filter params (role.id=43a16638caac4dd8acae2987c010b48a, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f, user.id=f32656e3430d43b4af5f4f34238f26ac) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:14.801129 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20870389-aac4-460a-8450-9f5189ab27aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:14.804148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20870389-aac4-460a-8450-9f5189ab27aa None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:14.812185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 125/250] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:14 2022] GET /identity/v3/role_assignments?user.id=f32656e3430d43b4af5f4f34238f26ac&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=43a16638caac4dd8acae2987c010b48a => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:14.869273 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd37d5-5d0c-4d9b-8250-5e15cb10cfe7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:14.871032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:14.906965 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a250d-e41a-4b3b-a7fc-6760f47a1464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:14.908108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a250d-e41a-4b3b-a7fc-6760f47a1464 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:14.908531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a250d-e41a-4b3b-a7fc-6760f47a1464 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:14.908976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a250d-e41a-4b3b-a7fc-6760f47a1464 None admin] PATH_INFO: `/v3/users/83bfb2eb1a1f4b4eb9d792f87d3ed25c` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:14.910479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a250d-e41a-4b3b-a7fc-6760f47a1464 None admin] RBAC: Authorizing `identity:get_user(user_id=83bfb2eb1a1f4b4eb9d792f87d3ed25c)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:14.912156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a250d-e41a-4b3b-a7fc-6760f47a1464 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:14.914957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 126/251] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:14 2022] GET /identity/v3/users/83bfb2eb1a1f4b4eb9d792f87d3ed25c => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:14.922351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20870389-aac4-460a-8450-9f5189ab27aa None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:14.994175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:15.036708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20870389-aac4-460a-8450-9f5189ab27aa None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.072362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35348b12-7351-4fc7-bb1f-ff25cdf9a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.073751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35348b12-7351-4fc7-bb1f-ff25cdf9a9fc None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.074621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35348b12-7351-4fc7-bb1f-ff25cdf9a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.075082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35348b12-7351-4fc7-bb1f-ff25cdf9a9fc None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.076402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35348b12-7351-4fc7-bb1f-ff25cdf9a9fc None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:15.078011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35348b12-7351-4fc7-bb1f-ff25cdf9a9fc None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:15.079836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 126/252] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:14 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:15.088669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a250d-e41a-4b3b-a7fc-6760f47a1464 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:15.231978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a250d-e41a-4b3b-a7fc-6760f47a1464 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.273101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3923d-2bcf-44ca-9b33-bb65f3a3a0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.274027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3923d-2bcf-44ca-9b33-bb65f3a3a0a7 None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.274545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3923d-2bcf-44ca-9b33-bb65f3a3a0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.274545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3923d-2bcf-44ca-9b33-bb65f3a3a0a7 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/users/83bfb2eb1a1f4b4eb9d792f87d3ed25c/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.279196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3923d-2bcf-44ca-9b33-bb65f3a3a0a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4088a75edb3f4c128c5f08a69f835d4f, user_id=83bfb2eb1a1f4b4eb9d792f87d3ed25c, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:15.280722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a3923d-2bcf-44ca-9b33-bb65f3a3a0a7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:15.366827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0264c5e-0056-4424-acce-5b3de91ec766 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.366827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0264c5e-0056-4424-acce-5b3de91ec766 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.366827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0264c5e-0056-4424-acce-5b3de91ec766 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.366827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 127/253] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:37:15.373607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78602ee7-a3ff-4e75-8c9a-b706c34690d8 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.373897 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78602ee7-a3ff-4e75-8c9a-b706c34690d8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.374184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78602ee7-a3ff-4e75-8c9a-b706c34690d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.374515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 127/254] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:37:15 2022] PUT /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/users/83bfb2eb1a1f4b4eb9d792f87d3ed25c/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:15.395042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-78602ee7-a3ff-4e75-8c9a-b706c34690d8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:15.396330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-78602ee7-a3ff-4e75-8c9a-b706c34690d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:15.480732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 128/255] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:15.516611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3923d-2bcf-44ca-9b33-bb65f3a3a0a7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:15.563764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:15.657178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3923d-2bcf-44ca-9b33-bb65f3a3a0a7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.698494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923c112-77f9-44b4-97f8-214c8921a681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.699297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923c112-77f9-44b4-97f8-214c8921a681 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.699582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923c112-77f9-44b4-97f8-214c8921a681 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.699856 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923c112-77f9-44b4-97f8-214c8921a681 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.711910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923c112-77f9-44b4-97f8-214c8921a681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-8923c112-77f9-44b4-97f8-214c8921a681 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:15.719756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:15.720630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:15.721456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 128/256] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:37:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:15.729144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78602ee7-a3ff-4e75-8c9a-b706c34690d8 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:15.823850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:15.869227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78602ee7-a3ff-4e75-8c9a-b706c34690d8 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:15.907482 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cfd2d-3eab-4495-8b41-da2eb667d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:15.908152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cfd2d-3eab-4495-8b41-da2eb667d620 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:15.908437 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cfd2d-3eab-4495-8b41-da2eb667d620 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:15.908708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cfd2d-3eab-4495-8b41-da2eb667d620 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:15.909477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cfd2d-3eab-4495-8b41-da2eb667d620 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:15.909787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cfd2d-3eab-4495-8b41-da2eb667d620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:15.911173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cfd2d-3eab-4495-8b41-da2eb667d620 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:15.924726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 129/257] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:15.930193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923c112-77f9-44b4-97f8-214c8921a681 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:16.033876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4b55a-6b86-4279-a307-fad641e86acb None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:16.034213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4b55a-6b86-4279-a307-fad641e86acb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:16.034439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4b55a-6b86-4279-a307-fad641e86acb None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:16.035953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 130/258] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:16.042402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bdea6-5c78-4535-81f7-03691410c0b4 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:16.042694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bdea6-5c78-4535-81f7-03691410c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:16.042977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bdea6-5c78-4535-81f7-03691410c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:16.063072 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-e02bdea6-5c78-4535-81f7-03691410c0b4 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:16.064558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e02bdea6-5c78-4535-81f7-03691410c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:16.073397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:16.090619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923c112-77f9-44b4-97f8-214c8921a681 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:16.150116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083f24b-9a07-42b2-b39c-680a323e0a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:16.151016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083f24b-9a07-42b2-b39c-680a323e0a43 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:16.151299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083f24b-9a07-42b2-b39c-680a323e0a43 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:16.151576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083f24b-9a07-42b2-b39c-680a323e0a43 None admin] PATH_INFO: `/v3/users/c9ae878d0d1841e79280707ee29a1b60` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:16.154034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083f24b-9a07-42b2-b39c-680a323e0a43 None admin] RBAC: Authorizing `identity:get_user(user_id=c9ae878d0d1841e79280707ee29a1b60)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:16.156510 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083f24b-9a07-42b2-b39c-680a323e0a43 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:16.158346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 129/259] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:15 2022] GET /identity/v3/users/c9ae878d0d1841e79280707ee29a1b60 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:16.167947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083f24b-9a07-42b2-b39c-680a323e0a43 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:16.195745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 131/260] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:16.247033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bdea6-5c78-4535-81f7-03691410c0b4 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:16.318393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083f24b-9a07-42b2-b39c-680a323e0a43 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:16.338458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:16.390900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e54915-5337-4c1e-af00-59d859c8b0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:16.394119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e54915-5337-4c1e-af00-59d859c8b0bd None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:16.394431 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e54915-5337-4c1e-af00-59d859c8b0bd None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:16.394714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e54915-5337-4c1e-af00-59d859c8b0bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:16.448069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bdea6-5c78-4535-81f7-03691410c0b4 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:16.451776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:16.509075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e54915-5337-4c1e-af00-59d859c8b0bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:16.510813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a5b08-2d20-4de4-aec4-84bc42517912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:16.510939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e54915-5337-4c1e-af00-59d859c8b0bd None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:16.511634 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a5b08-2d20-4de4-aec4-84bc42517912 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:16.511916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a5b08-2d20-4de4-aec4-84bc42517912 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:16.512186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a5b08-2d20-4de4-aec4-84bc42517912 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:16.530293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a5b08-2d20-4de4-aec4-84bc42517912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-7d4a5b08-2d20-4de4-aec4-84bc42517912 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:16.546658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:16.547539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:16.548313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 132/261] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:37:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:16.556795 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a5b08-2d20-4de4-aec4-84bc42517912 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:16.619483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 130/262] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:16.626216 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e54915-5337-4c1e-af00-59d859c8b0bd None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:16.716009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:16.778929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e54915-5337-4c1e-af00-59d859c8b0bd None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:16.811209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a5b08-2d20-4de4-aec4-84bc42517912 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:16.835311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d817a7-8eab-4908-9588-5f3e091ffe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:16.835689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:16.836191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d817a7-8eab-4908-9588-5f3e091ffe1f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:16.836476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d817a7-8eab-4908-9588-5f3e091ffe1f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:16.836755 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d817a7-8eab-4908-9588-5f3e091ffe1f None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:16.838022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d817a7-8eab-4908-9588-5f3e091ffe1f None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:16.839502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d817a7-8eab-4908-9588-5f3e091ffe1f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:16.845636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 131/263] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:16 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:16.847096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d817a7-8eab-4908-9588-5f3e091ffe1f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:16.954648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc1bb3-bd28-4951-a548-41353293007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:16.955262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc1bb3-bd28-4951-a548-41353293007b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:16.955553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc1bb3-bd28-4951-a548-41353293007b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:16.955831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc1bb3-bd28-4951-a548-41353293007b None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:16.956586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc1bb3-bd28-4951-a548-41353293007b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:16.956896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc1bb3-bd28-4951-a548-41353293007b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:16.958326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc1bb3-bd28-4951-a548-41353293007b None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:16.976345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 133/264] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:16.986240 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d817a7-8eab-4908-9588-5f3e091ffe1f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:16.986500 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc1bb3-bd28-4951-a548-41353293007b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:17.033881 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:17.054337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634057-364c-493e-94a5-7962033aa405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:17.055157 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634057-364c-493e-94a5-7962033aa405 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.055451 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634057-364c-493e-94a5-7962033aa405 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.055733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13634057-364c-493e-94a5-7962033aa405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.056690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13634057-364c-493e-94a5-7962033aa405 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f, user.id=c9ae878d0d1841e79280707ee29a1b60) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:17.057004 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13634057-364c-493e-94a5-7962033aa405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:17.058559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13634057-364c-493e-94a5-7962033aa405 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:17.072310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 132/265] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:16 2022] GET /identity/v3/role_assignments?user.id=c9ae878d0d1841e79280707ee29a1b60&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:17.177839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc1bb3-bd28-4951-a548-41353293007b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:17.221022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55038ae7-ccca-4eea-a5d4-8390c71415f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:17.222246 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55038ae7-ccca-4eea-a5d4-8390c71415f2 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.222894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55038ae7-ccca-4eea-a5d4-8390c71415f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.223361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55038ae7-ccca-4eea-a5d4-8390c71415f2 None admin] PATH_INFO: `/v3/users/c9ae878d0d1841e79280707ee29a1b60` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.230034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55038ae7-ccca-4eea-a5d4-8390c71415f2 None admin] RBAC: Authorizing `identity:get_user(user_id=c9ae878d0d1841e79280707ee29a1b60)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:17.231780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55038ae7-ccca-4eea-a5d4-8390c71415f2 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:17.233687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 134/266] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:16 2022] GET /identity/v3/users/c9ae878d0d1841e79280707ee29a1b60 => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:17.239560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634057-364c-493e-94a5-7962033aa405 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:17.394448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13634057-364c-493e-94a5-7962033aa405 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:17.436183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bdb43-50c7-4f50-b408-b5146ad8b4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:17.436833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdb43-50c7-4f50-b408-b5146ad8b4a0 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.437114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdb43-50c7-4f50-b408-b5146ad8b4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.437415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdb43-50c7-4f50-b408-b5146ad8b4a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.509169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:17.525887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bdb43-50c7-4f50-b408-b5146ad8b4a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:17.527409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bdb43-50c7-4f50-b408-b5146ad8b4a0 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:17.564073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163f116-053a-4811-befc-0d9345ed3f11 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.564584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163f116-053a-4811-befc-0d9345ed3f11 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.564659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163f116-053a-4811-befc-0d9345ed3f11 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.566346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 135/267] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:17.584610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75a1f7-574c-4004-8d78-39ba2b756e19 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.585916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75a1f7-574c-4004-8d78-39ba2b756e19 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.586205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75a1f7-574c-4004-8d78-39ba2b756e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.622734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0a75a1f7-574c-4004-8d78-39ba2b756e19 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:17.623967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0a75a1f7-574c-4004-8d78-39ba2b756e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:17.662606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 133/268] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:17.664091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ee28e-46fa-4ebc-95e3-0aacc65bafef None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.664385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ee28e-46fa-4ebc-95e3-0aacc65bafef None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.664661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ee28e-46fa-4ebc-95e3-0aacc65bafef None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.666243 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 134/269] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:17.672042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88dc71-d8c9-4b69-9266-82e306ffee6a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.672332 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88dc71-d8c9-4b69-9266-82e306ffee6a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.672615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88dc71-d8c9-4b69-9266-82e306ffee6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.693719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-df88dc71-d8c9-4b69-9266-82e306ffee6a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:17.695518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:17.696993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-df88dc71-d8c9-4b69-9266-82e306ffee6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:17.732315 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 136/270] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:17.733668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75a1f7-574c-4004-8d78-39ba2b756e19 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:17.780209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 135/271] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:17.782170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d028d58-4107-4c7e-811e-6fdd78cb2e8f None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.782789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d028d58-4107-4c7e-811e-6fdd78cb2e8f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.783632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d028d58-4107-4c7e-811e-6fdd78cb2e8f None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.785819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 136/272] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:17.787017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d028d58-4107-4c7e-811e-6fdd78cb2e8f None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:17.815966 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:17.845333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75a1f7-574c-4004-8d78-39ba2b756e19 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:17.881071 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8aa51-5d40-4913-a5a7-bd7b62792a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:17.882597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8aa51-5d40-4913-a5a7-bd7b62792a36 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.882884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8aa51-5d40-4913-a5a7-bd7b62792a36 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.883161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8aa51-5d40-4913-a5a7-bd7b62792a36 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.884348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8aa51-5d40-4913-a5a7-bd7b62792a36 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:17.885827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8aa51-5d40-4913-a5a7-bd7b62792a36 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:17.887515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 137/273] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:17 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:17.888641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0189ad-b3cb-4665-85b4-e96f49180f98 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.888924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0189ad-b3cb-4665-85b4-e96f49180f98 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.889208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0189ad-b3cb-4665-85b4-e96f49180f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.910279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d028d58-4107-4c7e-811e-6fdd78cb2e8f None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:17.910644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ef0189ad-b3cb-4665-85b4-e96f49180f98 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:17.911833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ef0189ad-b3cb-4665-85b4-e96f49180f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:17.948881 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81b191-1ec9-43e8-8693-8b355ab70cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:17.949711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81b191-1ec9-43e8-8693-8b355ab70cf0 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:17.949998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81b191-1ec9-43e8-8693-8b355ab70cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:17.950328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f81b191-1ec9-43e8-8693-8b355ab70cf0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:17.952331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:17.960614 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f81b191-1ec9-43e8-8693-8b355ab70cf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-9f81b191-1ec9-43e8-8693-8b355ab70cf0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:17.968358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:17.969285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:17.970072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 137/274] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:17.970720 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81b191-1ec9-43e8-8693-8b355ab70cf0 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:17.993015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 138/275] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:17.994202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0189ad-b3cb-4665-85b4-e96f49180f98 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:18.072008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:18.092228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f81b191-1ec9-43e8-8693-8b355ab70cf0 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:18.099744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0189ad-b3cb-4665-85b4-e96f49180f98 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:18.128206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3174d-6ed6-4c19-86a2-4852ac95d10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:18.128991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3174d-6ed6-4c19-86a2-4852ac95d10b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:18.129303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3174d-6ed6-4c19-86a2-4852ac95d10b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:18.129582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3174d-6ed6-4c19-86a2-4852ac95d10b None admin] PATH_INFO: `/v3/domains/default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:18.130734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3174d-6ed6-4c19-86a2-4852ac95d10b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:18.132742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3174d-6ed6-4c19-86a2-4852ac95d10b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:18.134649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 138/276] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:18.135297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3174d-6ed6-4c19-86a2-4852ac95d10b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:18.135496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9412b-641c-4aab-974f-611a26373643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:18.136138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9412b-641c-4aab-974f-611a26373643 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:18.136418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9412b-641c-4aab-974f-611a26373643 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:18.136689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9412b-641c-4aab-974f-611a26373643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:18.137530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9412b-641c-4aab-974f-611a26373643 None admin] RBAC: Adding query filter params (role.id=5f24daf2ba2b4f8fab95c87cefb52231, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f, user.id=c9ae878d0d1841e79280707ee29a1b60) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:18.137850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9412b-641c-4aab-974f-611a26373643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:18.139353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9412b-641c-4aab-974f-611a26373643 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:18.145371 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 139/277] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:17 2022] GET /identity/v3/role_assignments?user.id=c9ae878d0d1841e79280707ee29a1b60&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=5f24daf2ba2b4f8fab95c87cefb52231 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:18.146130 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9412b-641c-4aab-974f-611a26373643 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:18.190295 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:18.269484 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9412b-641c-4aab-974f-611a26373643 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:18.269683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3174d-6ed6-4c19-86a2-4852ac95d10b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:18.309451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faa420-2a98-4185-8f94-11421078900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:18.309907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ddc2-6709-464c-9f6e-d7aef64cac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:18.310290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faa420-2a98-4185-8f94-11421078900b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:18.310572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faa420-2a98-4185-8f94-11421078900b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:18.310741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ddc2-6709-464c-9f6e-d7aef64cac3b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:18.310833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faa420-2a98-4185-8f94-11421078900b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:18.311147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ddc2-6709-464c-9f6e-d7aef64cac3b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:18.311614 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ddc2-6709-464c-9f6e-d7aef64cac3b None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:18.312382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ddc2-6709-464c-9f6e-d7aef64cac3b None admin] RBAC: Adding query filter params (name=reader) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:18.312804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ddc2-6709-464c-9f6e-d7aef64cac3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:18.314347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ddc2-6709-464c-9f6e-d7aef64cac3b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:18.322903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5faa420-2a98-4185-8f94-11421078900b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:18.324414 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 139/278] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:18.325119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ddc2-6709-464c-9f6e-d7aef64cac3b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-a5faa420-2a98-4185-8f94-11421078900b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:18.336873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:18.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:18.338640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:18.339236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:18.339304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 140/279] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:18.340274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faa420-2a98-4185-8f94-11421078900b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:18.462160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ddc2-6709-464c-9f6e-d7aef64cac3b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:18.465605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:18.472444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faa420-2a98-4185-8f94-11421078900b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:18.498294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98b3cb-ee18-4fe2-b0c8-3eb2289fd700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:18.498968 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98b3cb-ee18-4fe2-b0c8-3eb2289fd700 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:18.499258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98b3cb-ee18-4fe2-b0c8-3eb2289fd700 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:18.499534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98b3cb-ee18-4fe2-b0c8-3eb2289fd700 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:18.500229 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98b3cb-ee18-4fe2-b0c8-3eb2289fd700 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:18.501739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98b3cb-ee18-4fe2-b0c8-3eb2289fd700 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:18.508946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2425260-a32d-4a7f-9d19-13c63c9f5326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:18.510086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2425260-a32d-4a7f-9d19-13c63c9f5326 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:18.510578 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2425260-a32d-4a7f-9d19-13c63c9f5326 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:18.511038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2425260-a32d-4a7f-9d19-13c63c9f5326 None admin] PATH_INFO: `/v3/users/f32656e3430d43b4af5f4f34238f26ac` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:18.512244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2425260-a32d-4a7f-9d19-13c63c9f5326 None admin] RBAC: Authorizing `identity:get_user(user_id=f32656e3430d43b4af5f4f34238f26ac)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:18.513749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2425260-a32d-4a7f-9d19-13c63c9f5326 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:18.515385 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 141/280] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:18 2022] GET /identity/v3/users/f32656e3430d43b4af5f4f34238f26ac => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:18.516147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2425260-a32d-4a7f-9d19-13c63c9f5326 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:18.615844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2425260-a32d-4a7f-9d19-13c63c9f5326 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:18.629669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 140/281] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:37:18 2022] POST /identity/v3/users => generated 293 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:18.629669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98b3cb-ee18-4fe2-b0c8-3eb2289fd700 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:18.671997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf3728-df94-44b9-a8f2-fcaf8c700733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:18.673228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf3728-df94-44b9-a8f2-fcaf8c700733 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:18.673552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf3728-df94-44b9-a8f2-fcaf8c700733 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:18.674157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf3728-df94-44b9-a8f2-fcaf8c700733 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:18.674926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf3728-df94-44b9-a8f2-fcaf8c700733 None admin] RBAC: Adding query filter params (name=member) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:18.675556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf3728-df94-44b9-a8f2-fcaf8c700733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:18.677836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcf3728-df94-44b9-a8f2-fcaf8c700733 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:18.691553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 142/282] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:18.698132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf3728-df94-44b9-a8f2-fcaf8c700733 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:18.734645 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:18.739798 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:18.776501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98b3cb-ee18-4fe2-b0c8-3eb2289fd700 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:18.815998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a057b67-6605-488c-91f1-8f5e4e0bae75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:18.817199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a057b67-6605-488c-91f1-8f5e4e0bae75 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:18.817693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a057b67-6605-488c-91f1-8f5e4e0bae75 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:18.818148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a057b67-6605-488c-91f1-8f5e4e0bae75 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:18.819667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a057b67-6605-488c-91f1-8f5e4e0bae75 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:18.821384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a057b67-6605-488c-91f1-8f5e4e0bae75 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:18.823625 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 141/283] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:18 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:18.828931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a057b67-6605-488c-91f1-8f5e4e0bae75 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:18.841711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcf3728-df94-44b9-a8f2-fcaf8c700733 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:18.889957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81596e-48a9-489f-9f6c-e54531ea9d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:18.891956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81596e-48a9-489f-9f6c-e54531ea9d0b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:18.892584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81596e-48a9-489f-9f6c-e54531ea9d0b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:18.893173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d81596e-48a9-489f-9f6c-e54531ea9d0b None admin] PATH_INFO: `/v3/users/d75c0c9bb2b941829a30b68cbdf557be` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:18.912532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81596e-48a9-489f-9f6c-e54531ea9d0b None admin] RBAC: Authorizing `identity:get_user(user_id=d75c0c9bb2b941829a30b68cbdf557be)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:18.915183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d81596e-48a9-489f-9f6c-e54531ea9d0b None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:18.922248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 143/284] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:18 2022] GET /identity/v3/users/d75c0c9bb2b941829a30b68cbdf557be => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:18.925658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81596e-48a9-489f-9f6c-e54531ea9d0b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:18.958084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a057b67-6605-488c-91f1-8f5e4e0bae75 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:19.007695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded5296-1101-4ff8-bfe7-21c4c46eae9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:19.008918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5296-1101-4ff8-bfe7-21c4c46eae9c None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:19.009210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5296-1101-4ff8-bfe7-21c4c46eae9c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:19.009529 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded5296-1101-4ff8-bfe7-21c4c46eae9c None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/users/f32656e3430d43b4af5f4f34238f26ac/roles/43a16638caac4dd8acae2987c010b48a` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:19.011527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded5296-1101-4ff8-bfe7-21c4c46eae9c None admin] RBAC: Authorizing `identity:create_grant(project_id=4088a75edb3f4c128c5f08a69f835d4f, user_id=f32656e3430d43b4af5f4f34238f26ac, role_id=43a16638caac4dd8acae2987c010b48a)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:19.013083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded5296-1101-4ff8-bfe7-21c4c46eae9c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:19.016850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:19.025664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:19.053875 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d81596e-48a9-489f-9f6c-e54531ea9d0b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:19.094142 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0d276-0753-42e8-9f0d-6648f5a7934c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:19.094350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 142/285] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:37:18 2022] PUT /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/users/f32656e3430d43b4af5f4f34238f26ac/roles/43a16638caac4dd8acae2987c010b48a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:19.097270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0d276-0753-42e8-9f0d-6648f5a7934c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:19.097270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0d276-0753-42e8-9f0d-6648f5a7934c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:19.097270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0d276-0753-42e8-9f0d-6648f5a7934c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:19.097588 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d735f-2d5f-49f0-9666-99b3b4aa6c07 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:19.097588 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d735f-2d5f-49f0-9666-99b3b4aa6c07 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:19.097588 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d735f-2d5f-49f0-9666-99b3b4aa6c07 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:19.097812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0d276-0753-42e8-9f0d-6648f5a7934c None admin] RBAC: Adding query filter params (role.id=0960434fd9da404ea96841eb4306eaae, user.id=d75c0c9bb2b941829a30b68cbdf557be, scope.system=all) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:19.097812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0d276-0753-42e8-9f0d-6648f5a7934c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:19.098095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 143/286] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:19.099302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0d276-0753-42e8-9f0d-6648f5a7934c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:19.104139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212db5b-aa25-4d0b-9a78-bec9dc4e7eb7 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:19.104520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212db5b-aa25-4d0b-9a78-bec9dc4e7eb7 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:19.104899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212db5b-aa25-4d0b-9a78-bec9dc4e7eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:19.107275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 144/287] 10.222.0.81 () {56 vars in 1308 bytes} [Thu Jun 16 18:37:18 2022] GET /identity/v3/role_assignments?user.id=d75c0c9bb2b941829a30b68cbdf557be&scope.system=all&role.id=0960434fd9da404ea96841eb4306eaae => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:19.154635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-e212db5b-aa25-4d0b-9a78-bec9dc4e7eb7 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:19.155966 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e212db5b-aa25-4d0b-9a78-bec9dc4e7eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:19.274534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 144/288] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:19.301877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0d276-0753-42e8-9f0d-6648f5a7934c None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:19.417371 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0d276-0753-42e8-9f0d-6648f5a7934c None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:19.452782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c59bd-d85a-4053-acab-2e2e871cddac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:19.453876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c59bd-d85a-4053-acab-2e2e871cddac None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:19.454305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c59bd-d85a-4053-acab-2e2e871cddac None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:19.454729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c59bd-d85a-4053-acab-2e2e871cddac None admin] PATH_INFO: `/v3/roles/member` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:19.465130 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c59bd-d85a-4053-acab-2e2e871cddac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-865c59bd-d85a-4053-acab-2e2e871cddac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:19.472997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:19.474085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:19.475079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:19.477515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 145/289] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:19.483029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212db5b-aa25-4d0b-9a78-bec9dc4e7eb7 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:19.600225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212db5b-aa25-4d0b-9a78-bec9dc4e7eb7 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:19.633926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:19.645732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a0fbf-1af7-44d8-b143-5f741dc9def0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:19.646616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0fbf-1af7-44d8-b143-5f741dc9def0 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:19.647051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0fbf-1af7-44d8-b143-5f741dc9def0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:19.647473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0fbf-1af7-44d8-b143-5f741dc9def0 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:19.648960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a0fbf-1af7-44d8-b143-5f741dc9def0 None admin] RBAC: Adding query filter params (name=member) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:19.649481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a0fbf-1af7-44d8-b143-5f741dc9def0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:19.651082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a0fbf-1af7-44d8-b143-5f741dc9def0 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:19.665910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 145/290] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:19.671629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c59bd-d85a-4053-acab-2e2e871cddac None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:19.684067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:19.821842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c59bd-d85a-4053-acab-2e2e871cddac None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:19.868024 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a079d22-b4df-45a0-966c-65a6ec81348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:19.869345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a079d22-b4df-45a0-966c-65a6ec81348f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:19.869830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a079d22-b4df-45a0-966c-65a6ec81348f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:19.870303 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a079d22-b4df-45a0-966c-65a6ec81348f None admin] PATH_INFO: `/v3/users/83bfb2eb1a1f4b4eb9d792f87d3ed25c` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:19.877947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a079d22-b4df-45a0-966c-65a6ec81348f None admin] RBAC: Authorizing `identity:get_user(user_id=83bfb2eb1a1f4b4eb9d792f87d3ed25c)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:19.879611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a079d22-b4df-45a0-966c-65a6ec81348f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:19.881824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 146/291] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:19 2022] GET /identity/v3/users/83bfb2eb1a1f4b4eb9d792f87d3ed25c => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:19.887645 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a0fbf-1af7-44d8-b143-5f741dc9def0 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:20.019217 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a0fbf-1af7-44d8-b143-5f741dc9def0 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:20.066881 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845cd7f-edc1-40cb-95bd-c5bb6f49b1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:20.067923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845cd7f-edc1-40cb-95bd-c5bb6f49b1b5 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:20.068416 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845cd7f-edc1-40cb-95bd-c5bb6f49b1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:20.068885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c845cd7f-edc1-40cb-95bd-c5bb6f49b1b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:20.153620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:20.172566 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845cd7f-edc1-40cb-95bd-c5bb6f49b1b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:20.174060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c845cd7f-edc1-40cb-95bd-c5bb6f49b1b5 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:20.316960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 146/292] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:20.323042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a079d22-b4df-45a0-966c-65a6ec81348f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:20.436678 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:20.470599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a079d22-b4df-45a0-966c-65a6ec81348f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:20.505170 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d3129-9d15-478d-a11d-06624b5076eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:20.506066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d3129-9d15-478d-a11d-06624b5076eb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:20.506350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d3129-9d15-478d-a11d-06624b5076eb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:20.506625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d3129-9d15-478d-a11d-06624b5076eb None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:20.507823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d3129-9d15-478d-a11d-06624b5076eb None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:20.509291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d3129-9d15-478d-a11d-06624b5076eb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:20.514921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 147/293] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:20 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:20.517686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845cd7f-edc1-40cb-95bd-c5bb6f49b1b5 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:20.613777 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:20.629229 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c845cd7f-edc1-40cb-95bd-c5bb6f49b1b5 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:20.673438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad24904-0462-4a9d-aac8-495ed7f44ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:20.674113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad24904-0462-4a9d-aac8-495ed7f44ebb None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:20.674397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad24904-0462-4a9d-aac8-495ed7f44ebb None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:20.674671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad24904-0462-4a9d-aac8-495ed7f44ebb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:20.675501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad24904-0462-4a9d-aac8-495ed7f44ebb None admin] RBAC: Adding query filter params (role.id=0960434fd9da404ea96841eb4306eaae, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f, user.id=83bfb2eb1a1f4b4eb9d792f87d3ed25c) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:20.675988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad24904-0462-4a9d-aac8-495ed7f44ebb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:20.677311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad24904-0462-4a9d-aac8-495ed7f44ebb None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:20.690097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 147/294] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:20 2022] GET /identity/v3/role_assignments?user.id=83bfb2eb1a1f4b4eb9d792f87d3ed25c&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=0960434fd9da404ea96841eb4306eaae => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:21.394158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c893-be93-48c0-9d6f-dad0e0089ee5 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:21.394699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c893-be93-48c0-9d6f-dad0e0089ee5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:21.394699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c893-be93-48c0-9d6f-dad0e0089ee5 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:21.396340 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 148/295] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:21.401504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cc7e1-6ba5-4e17-8ebd-89e73b13272a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:21.401803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cc7e1-6ba5-4e17-8ebd-89e73b13272a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:21.402084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634cc7e1-6ba5-4e17-8ebd-89e73b13272a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:21.420115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-634cc7e1-6ba5-4e17-8ebd-89e73b13272a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:21.421424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-634cc7e1-6ba5-4e17-8ebd-89e73b13272a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:21.499953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 148/296] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:21.517066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae840ebd-ff59-4842-be79-fd99f72f7152 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:21.517543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae840ebd-ff59-4842-be79-fd99f72f7152 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:21.517833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae840ebd-ff59-4842-be79-fd99f72f7152 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:21.519212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 149/297] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:21.524975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca9e13-49e7-42be-9b23-dd12cec756ef None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:21.525466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca9e13-49e7-42be-9b23-dd12cec756ef None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:21.525542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca9e13-49e7-42be-9b23-dd12cec756ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:21.528442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae840ebd-ff59-4842-be79-fd99f72f7152 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:21.542827 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-8bca9e13-49e7-42be-9b23-dd12cec756ef None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:21.544187 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8bca9e13-49e7-42be-9b23-dd12cec756ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:21.633492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 149/298] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:21.640573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae840ebd-ff59-4842-be79-fd99f72f7152 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:21.654317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca9e13-49e7-42be-9b23-dd12cec756ef None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:21.681625 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:21.687772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45bfb4-8daa-4ad1-963d-c3b3a5969704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:21.689461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45bfb4-8daa-4ad1-963d-c3b3a5969704 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:21.689951 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45bfb4-8daa-4ad1-963d-c3b3a5969704 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:21.690420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45bfb4-8daa-4ad1-963d-c3b3a5969704 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:21.702430 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45bfb4-8daa-4ad1-963d-c3b3a5969704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-8f45bfb4-8daa-4ad1-963d-c3b3a5969704 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:21.710560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:21.711425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:21.712196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:21.714823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 150/299] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:21.720348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45bfb4-8daa-4ad1-963d-c3b3a5969704 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:21.723554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:21.800165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca9e13-49e7-42be-9b23-dd12cec756ef None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:21.833831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45bfb4-8daa-4ad1-963d-c3b3a5969704 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:21.838575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275eb44-aadb-472d-b879-5a1ccb9ecb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:21.839381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275eb44-aadb-472d-b879-5a1ccb9ecb5a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:21.839882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275eb44-aadb-472d-b879-5a1ccb9ecb5a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:21.839882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275eb44-aadb-472d-b879-5a1ccb9ecb5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:21.841076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275eb44-aadb-472d-b879-5a1ccb9ecb5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:21.842578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275eb44-aadb-472d-b879-5a1ccb9ecb5a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:21.844337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 150/300] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:21.847583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b9b51-3cd2-4556-8c85-bc6a1f019dac None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:21.848043 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b9b51-3cd2-4556-8c85-bc6a1f019dac None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:21.848326 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b9b51-3cd2-4556-8c85-bc6a1f019dac None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:21.849825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 151/301] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:21.850737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843ad15-7ed0-44b3-a891-4992967f6dfb None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:21.851045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843ad15-7ed0-44b3-a891-4992967f6dfb None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:21.851324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843ad15-7ed0-44b3-a891-4992967f6dfb None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:21.852544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 152/302] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:21.853224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843ad15-7ed0-44b3-a891-4992967f6dfb None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:21.876097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ebdb5-5e70-451e-b7fe-17ee6ede0085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:21.876757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ebdb5-5e70-451e-b7fe-17ee6ede0085 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:21.877036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ebdb5-5e70-451e-b7fe-17ee6ede0085 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:21.877340 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ebdb5-5e70-451e-b7fe-17ee6ede0085 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:21.878069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ebdb5-5e70-451e-b7fe-17ee6ede0085 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:21.878369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ebdb5-5e70-451e-b7fe-17ee6ede0085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:21.879738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ebdb5-5e70-451e-b7fe-17ee6ede0085 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:21.888260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 151/303] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:21.891173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583089c-3c79-4ae0-8c3e-0f3cfe8b2d9c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:21.891463 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583089c-3c79-4ae0-8c3e-0f3cfe8b2d9c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:21.891740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583089c-3c79-4ae0-8c3e-0f3cfe8b2d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:21.908841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6583089c-3c79-4ae0-8c3e-0f3cfe8b2d9c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:21.910059 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6583089c-3c79-4ae0-8c3e-0f3cfe8b2d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:21.932812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:21.955779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843ad15-7ed0-44b3-a891-4992967f6dfb None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:21.970257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:21.978128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 152/304] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:21.979412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1706f-1e3c-4ee3-a579-1b75fe1f09c8 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:21.979701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1706f-1e3c-4ee3-a579-1b75fe1f09c8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:21.979969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1706f-1e3c-4ee3-a579-1b75fe1f09c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:21.996403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-b1a1706f-1e3c-4ee3-a579-1b75fe1f09c8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:21.997626 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b1a1706f-1e3c-4ee3-a579-1b75fe1f09c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:22.000752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051bfbb-e2e4-45a4-b044-77249651267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:22.001455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051bfbb-e2e4-45a4-b044-77249651267e None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:22.001737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051bfbb-e2e4-45a4-b044-77249651267e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:22.002010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051bfbb-e2e4-45a4-b044-77249651267e None admin] PATH_INFO: `/v3/groups` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:22.002686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051bfbb-e2e4-45a4-b044-77249651267e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:22.004112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051bfbb-e2e4-45a4-b044-77249651267e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:22.059567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 153/305] 10.222.0.81 () {60 vars in 1134 bytes} [Thu Jun 16 18:37:21 2022] POST /identity/v3/groups => generated 229 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:22.060418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051bfbb-e2e4-45a4-b044-77249651267e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:22.062773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 153/306] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:22.064038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1706f-1e3c-4ee3-a579-1b75fe1f09c8 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:22.187755 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051bfbb-e2e4-45a4-b044-77249651267e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:22.203429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1706f-1e3c-4ee3-a579-1b75fe1f09c8 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:22.229221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf499e67-1a82-4a5c-8220-c56993c87997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:22.229848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:22.230129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf499e67-1a82-4a5c-8220-c56993c87997 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:22.230415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf499e67-1a82-4a5c-8220-c56993c87997 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:22.230784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf499e67-1a82-4a5c-8220-c56993c87997 None admin] PATH_INFO: `/v3/users/da63b34c457e44b1834608951ba16839` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:22.239190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:22.242396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978309c-26b5-4993-8790-0618db02d631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:22.243665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978309c-26b5-4993-8790-0618db02d631 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:22.243961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978309c-26b5-4993-8790-0618db02d631 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:22.244244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978309c-26b5-4993-8790-0618db02d631 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:22.246867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf499e67-1a82-4a5c-8220-c56993c87997 None admin] RBAC: Authorizing `identity:get_user(user_id=da63b34c457e44b1834608951ba16839)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:22.248348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf499e67-1a82-4a5c-8220-c56993c87997 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:22.250586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 154/307] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:22 2022] GET /identity/v3/users/da63b34c457e44b1834608951ba16839 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:22.251330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf499e67-1a82-4a5c-8220-c56993c87997 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:22.257885 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978309c-26b5-4993-8790-0618db02d631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-0978309c-26b5-4993-8790-0618db02d631 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:22.265723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:22.266620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:22.267405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 154/308] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:22.268047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978309c-26b5-4993-8790-0618db02d631 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:22.372990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf499e67-1a82-4a5c-8220-c56993c87997 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:22.388265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978309c-26b5-4993-8790-0618db02d631 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:22.410010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bc71b-2e97-472f-a8d9-5884bfe18ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:22.410816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bc71b-2e97-472f-a8d9-5884bfe18ec0 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:22.411100 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bc71b-2e97-472f-a8d9-5884bfe18ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:22.411374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bc71b-2e97-472f-a8d9-5884bfe18ec0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:22.422594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bc71b-2e97-472f-a8d9-5884bfe18ec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:22.423977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f3b08-54ef-4d29-b114-88e0c888c17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:22.424630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f3b08-54ef-4d29-b114-88e0c888c17a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:22.424916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f3b08-54ef-4d29-b114-88e0c888c17a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:22.425192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f3b08-54ef-4d29-b114-88e0c888c17a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:22.426069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f3b08-54ef-4d29-b114-88e0c888c17a None admin] RBAC: Adding query filter params (role.id=43a16638caac4dd8acae2987c010b48a, user.id=da63b34c457e44b1834608951ba16839, scope.system=all) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:22.426411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f3b08-54ef-4d29-b114-88e0c888c17a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:22.427846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f3b08-54ef-4d29-b114-88e0c888c17a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-dd3bc71b-2e97-472f-a8d9-5884bfe18ec0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:22.430351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:22.431253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:22.432037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 155/309] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:22.433085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bc71b-2e97-472f-a8d9-5884bfe18ec0 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:22.438770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 155/310] 10.222.0.81 () {56 vars in 1308 bytes} [Thu Jun 16 18:37:22 2022] GET /identity/v3/role_assignments?user.id=da63b34c457e44b1834608951ba16839&scope.system=all&role.id=43a16638caac4dd8acae2987c010b48a => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:22.439885 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f3b08-54ef-4d29-b114-88e0c888c17a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:22.500257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:22.505730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:22.570472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bc71b-2e97-472f-a8d9-5884bfe18ec0 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:22.572503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f3b08-54ef-4d29-b114-88e0c888c17a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:22.616387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48865032-ca45-4c11-a888-765844b511ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:22.617514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48865032-ca45-4c11-a888-765844b511ab None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:22.617981 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48865032-ca45-4c11-a888-765844b511ab None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:22.618434 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48865032-ca45-4c11-a888-765844b511ab None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:22.619384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48865032-ca45-4c11-a888-765844b511ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:22.619778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a39858-b4de-4725-ae06-3b1629aa0089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:22.620110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48865032-ca45-4c11-a888-765844b511ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:22.620451 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a39858-b4de-4725-ae06-3b1629aa0089 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:22.620736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a39858-b4de-4725-ae06-3b1629aa0089 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:22.621011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a39858-b4de-4725-ae06-3b1629aa0089 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:22.621953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a39858-b4de-4725-ae06-3b1629aa0089 None admin] RBAC: Adding query filter params (name=member) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:22.622391 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a39858-b4de-4725-ae06-3b1629aa0089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:22.623660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48865032-ca45-4c11-a888-765844b511ab None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:22.623835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a39858-b4de-4725-ae06-3b1629aa0089 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:22.633488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 156/311] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:22.637686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 156/312] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:22.638886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a39858-b4de-4725-ae06-3b1629aa0089 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:22.639978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48865032-ca45-4c11-a888-765844b511ab None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:22.746159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48865032-ca45-4c11-a888-765844b511ab None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:22.748744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a39858-b4de-4725-ae06-3b1629aa0089 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:22.757941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:22.767688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:22.782941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae3bdb-0582-45c5-a821-e2eb79a8a581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:22.783935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae3bdb-0582-45c5-a821-e2eb79a8a581 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:22.784220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae3bdb-0582-45c5-a821-e2eb79a8a581 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:22.784497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae3bdb-0582-45c5-a821-e2eb79a8a581 None admin] PATH_INFO: `/v3/users/d75c0c9bb2b941829a30b68cbdf557be` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:22.785757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae3bdb-0582-45c5-a821-e2eb79a8a581 None admin] RBAC: Authorizing `identity:get_user(user_id=d75c0c9bb2b941829a30b68cbdf557be)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:22.787445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae3bdb-0582-45c5-a821-e2eb79a8a581 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:22.788070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd19e7-34a6-4c4d-b8b6-bef8b4cea139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:22.790012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd19e7-34a6-4c4d-b8b6-bef8b4cea139 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:22.790547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd19e7-34a6-4c4d-b8b6-bef8b4cea139 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:22.791128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd19e7-34a6-4c4d-b8b6-bef8b4cea139 None admin] PATH_INFO: `/v3/users/f32656e3430d43b4af5f4f34238f26ac` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:22.792583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd19e7-34a6-4c4d-b8b6-bef8b4cea139 None admin] RBAC: Authorizing `identity:get_user(user_id=f32656e3430d43b4af5f4f34238f26ac)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:22.793580 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 157/313] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:22 2022] GET /identity/v3/users/d75c0c9bb2b941829a30b68cbdf557be => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:22.794688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd19e7-34a6-4c4d-b8b6-bef8b4cea139 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:22.795453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae3bdb-0582-45c5-a821-e2eb79a8a581 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:22.797291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 157/314] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:22 2022] GET /identity/v3/users/f32656e3430d43b4af5f4f34238f26ac => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:22.802116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd19e7-34a6-4c4d-b8b6-bef8b4cea139 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:22.899893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae3bdb-0582-45c5-a821-e2eb79a8a581 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:22.914684 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd19e7-34a6-4c4d-b8b6-bef8b4cea139 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:22.935886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11666e-2eec-4572-a965-0531e867a412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:22.936776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11666e-2eec-4572-a965-0531e867a412 None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:22.937085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11666e-2eec-4572-a965-0531e867a412 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:22.937369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11666e-2eec-4572-a965-0531e867a412 None admin] PATH_INFO: `/v3/system/users/d75c0c9bb2b941829a30b68cbdf557be/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:22.938842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11666e-2eec-4572-a965-0531e867a412 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d75c0c9bb2b941829a30b68cbdf557be, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:22.940198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11666e-2eec-4572-a965-0531e867a412 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:22.952899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94a9cb-0a03-41b8-b561-d77525b94366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:22.954223 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94a9cb-0a03-41b8-b561-d77525b94366 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:22.954707 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94a9cb-0a03-41b8-b561-d77525b94366 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:22.959629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94a9cb-0a03-41b8-b561-d77525b94366 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:23.012421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:23.027419 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94a9cb-0a03-41b8-b561-d77525b94366 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:23.028891 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94a9cb-0a03-41b8-b561-d77525b94366 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:23.049175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 158/315] 10.222.0.81 () {58 vars in 1334 bytes} [Thu Jun 16 18:37:22 2022] PUT /identity/v3/system/users/d75c0c9bb2b941829a30b68cbdf557be/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:37:23.156057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 158/316] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:37:23.161461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11666e-2eec-4572-a965-0531e867a412 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:23.212113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:23.276392 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11666e-2eec-4572-a965-0531e867a412 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:23.311842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f10c9-e7e6-4067-a823-d5a4c7909fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:23.312710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f10c9-e7e6-4067-a823-d5a4c7909fa5 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:23.312990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f10c9-e7e6-4067-a823-d5a4c7909fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:23.313277 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f10c9-e7e6-4067-a823-d5a4c7909fa5 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:23.314457 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f10c9-e7e6-4067-a823-d5a4c7909fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:23.315908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f10c9-e7e6-4067-a823-d5a4c7909fa5 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:23.317923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 159/317] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:23 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:23.324481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94a9cb-0a03-41b8-b561-d77525b94366 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:23.425777 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:23.441683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94a9cb-0a03-41b8-b561-d77525b94366 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:23.478097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94c0ea-8296-4858-b6d9-22d3038543f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:23.478999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94c0ea-8296-4858-b6d9-22d3038543f0 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:23.479464 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94c0ea-8296-4858-b6d9-22d3038543f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:23.479885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94c0ea-8296-4858-b6d9-22d3038543f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:23.480856 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94c0ea-8296-4858-b6d9-22d3038543f0 None admin] RBAC: Adding query filter params (role.id=43a16638caac4dd8acae2987c010b48a, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f, user.id=f32656e3430d43b4af5f4f34238f26ac) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:23.481515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94c0ea-8296-4858-b6d9-22d3038543f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:23.483165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94c0ea-8296-4858-b6d9-22d3038543f0 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:23.491022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 159/318] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:23 2022] GET /identity/v3/role_assignments?user.id=f32656e3430d43b4af5f4f34238f26ac&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=43a16638caac4dd8acae2987c010b48a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:24.221182 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d415f-7d73-4f8a-88d5-196d008c7f41 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:24.221738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d415f-7d73-4f8a-88d5-196d008c7f41 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:24.221738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d415f-7d73-4f8a-88d5-196d008c7f41 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:24.223246 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 160/319] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:24.228712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13540261-e879-474e-bf2a-f30638df8a21 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:24.229096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13540261-e879-474e-bf2a-f30638df8a21 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:24.229297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13540261-e879-474e-bf2a-f30638df8a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:24.247623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-13540261-e879-474e-bf2a-f30638df8a21 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:24.248820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-13540261-e879-474e-bf2a-f30638df8a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:24.316733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 160/320] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:24.340024 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d415f-7d73-4f8a-88d5-196d008c7f41 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:24.441197 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d415f-7d73-4f8a-88d5-196d008c7f41 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:24.452320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:24.475314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068500aa-e046-4ede-997d-48ed555c2550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:24.476035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068500aa-e046-4ede-997d-48ed555c2550 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:24.476451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068500aa-e046-4ede-997d-48ed555c2550 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:24.476703 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068500aa-e046-4ede-997d-48ed555c2550 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:24.477862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068500aa-e046-4ede-997d-48ed555c2550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:24.479280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068500aa-e046-4ede-997d-48ed555c2550 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:24.481024 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 161/321] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:24.486330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13540261-e879-474e-bf2a-f30638df8a21 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:24.563063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:24.584265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13540261-e879-474e-bf2a-f30638df8a21 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:24.620900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85efb66-9e6a-42a7-8897-f40aebbfc62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:24.621863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85efb66-9e6a-42a7-8897-f40aebbfc62f None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:24.622292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85efb66-9e6a-42a7-8897-f40aebbfc62f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:24.622709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85efb66-9e6a-42a7-8897-f40aebbfc62f None admin] PATH_INFO: `/v3/groups` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:24.623536 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85efb66-9e6a-42a7-8897-f40aebbfc62f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:24.625301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85efb66-9e6a-42a7-8897-f40aebbfc62f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:24.638344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9426-853f-4622-898c-3e2bd66ed8a6 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:24.638630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9426-853f-4622-898c-3e2bd66ed8a6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:24.638906 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da9426-853f-4622-898c-3e2bd66ed8a6 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:24.640293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 162/322] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:37:24.644982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab15ed2-6f22-4d37-bb70-229bfde873ad None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:24.645872 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab15ed2-6f22-4d37-bb70-229bfde873ad None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:24.645872 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab15ed2-6f22-4d37-bb70-229bfde873ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:24.662727 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-8ab15ed2-6f22-4d37-bb70-229bfde873ad None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:24.664287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8ab15ed2-6f22-4d37-bb70-229bfde873ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:24.712560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 161/323] 10.222.0.81 () {60 vars in 1134 bytes} [Thu Jun 16 18:37:24 2022] POST /identity/v3/groups => generated 226 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:24.735381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 163/324] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:24.759387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85efb66-9e6a-42a7-8897-f40aebbfc62f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:24.867123 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85efb66-9e6a-42a7-8897-f40aebbfc62f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:24.901246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c168b-801d-48a2-b619-dd14e226e67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:24.902066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c168b-801d-48a2-b619-dd14e226e67b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:24.902347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c168b-801d-48a2-b619-dd14e226e67b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:24.902618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c168b-801d-48a2-b619-dd14e226e67b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:24.910122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:24.914580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c168b-801d-48a2-b619-dd14e226e67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-d15c168b-801d-48a2-b619-dd14e226e67b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:24.922262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:24.925079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:24.926052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 162/325] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:24.929440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab15ed2-6f22-4d37-bb70-229bfde873ad None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:25.043789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab15ed2-6f22-4d37-bb70-229bfde873ad None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:25.053803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:25.078969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead72a7e-d542-4129-bcab-59109464fc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:25.079698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead72a7e-d542-4129-bcab-59109464fc2a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:25.079987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead72a7e-d542-4129-bcab-59109464fc2a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:25.080263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead72a7e-d542-4129-bcab-59109464fc2a None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:25.081018 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead72a7e-d542-4129-bcab-59109464fc2a None admin] RBAC: Adding query filter params (name=reader) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:25.081356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead72a7e-d542-4129-bcab-59109464fc2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:25.082755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead72a7e-d542-4129-bcab-59109464fc2a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:25.092354 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 164/326] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:25.097756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c168b-801d-48a2-b619-dd14e226e67b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:25.204748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c168b-801d-48a2-b619-dd14e226e67b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:25.240507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6672b0-bbc9-4e52-b509-088f6d571487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:25.241456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6672b0-bbc9-4e52-b509-088f6d571487 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:25.241746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6672b0-bbc9-4e52-b509-088f6d571487 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:25.242057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6672b0-bbc9-4e52-b509-088f6d571487 None admin] PATH_INFO: `/v3/users/da63b34c457e44b1834608951ba16839` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:25.243444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6672b0-bbc9-4e52-b509-088f6d571487 None admin] RBAC: Authorizing `identity:get_user(user_id=da63b34c457e44b1834608951ba16839)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:25.244963 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6672b0-bbc9-4e52-b509-088f6d571487 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:25.247043 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 163/327] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:25 2022] GET /identity/v3/users/da63b34c457e44b1834608951ba16839 => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:25.252836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead72a7e-d542-4129-bcab-59109464fc2a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:25.326632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9457c-cb84-49f5-b928-18137d28e0df None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:25.326949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9457c-cb84-49f5-b928-18137d28e0df None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:25.327193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9457c-cb84-49f5-b928-18137d28e0df None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:25.328573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 164/328] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:25.334213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd1cab-542c-4cd8-8403-215f1fb6ea5b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:25.334513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd1cab-542c-4cd8-8403-215f1fb6ea5b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:25.334786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd1cab-542c-4cd8-8403-215f1fb6ea5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:25.358568 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-16dd1cab-542c-4cd8-8403-215f1fb6ea5b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:25.360351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-16dd1cab-542c-4cd8-8403-215f1fb6ea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:25.376969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead72a7e-d542-4129-bcab-59109464fc2a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:25.422449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae38acd-4fe7-4a9a-a852-e01779a67e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:25.423328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae38acd-4fe7-4a9a-a852-e01779a67e22 None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:25.423613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae38acd-4fe7-4a9a-a852-e01779a67e22 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:25.423892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae38acd-4fe7-4a9a-a852-e01779a67e22 None admin] PATH_INFO: `/v3/system/users/da63b34c457e44b1834608951ba16839/roles/43a16638caac4dd8acae2987c010b48a` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:25.426399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae38acd-4fe7-4a9a-a852-e01779a67e22 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=da63b34c457e44b1834608951ba16839, role_id=43a16638caac4dd8acae2987c010b48a)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:25.427788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae38acd-4fe7-4a9a-a852-e01779a67e22 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:25.456511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:25.463635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 165/329] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:25.493474 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd1cab-542c-4cd8-8403-215f1fb6ea5b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:25.530009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 165/330] 10.222.0.81 () {58 vars in 1334 bytes} [Thu Jun 16 18:37:25 2022] PUT /identity/v3/system/users/da63b34c457e44b1834608951ba16839/roles/43a16638caac4dd8acae2987c010b48a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:25.614339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd1cab-542c-4cd8-8403-215f1fb6ea5b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:25.656918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee19c0-13ec-42a1-9a4d-c2e9c310c54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:25.657770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee19c0-13ec-42a1-9a4d-c2e9c310c54f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:25.658054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee19c0-13ec-42a1-9a4d-c2e9c310c54f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:25.658324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee19c0-13ec-42a1-9a4d-c2e9c310c54f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:25.670391 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee19c0-13ec-42a1-9a4d-c2e9c310c54f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-30ee19c0-13ec-42a1-9a4d-c2e9c310c54f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:25.678287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:25.679280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:25.680921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 166/331] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:25.685944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae38acd-4fe7-4a9a-a852-e01779a67e22 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:25.756372 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:25.829062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae38acd-4fe7-4a9a-a852-e01779a67e22 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:25.872264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a1443-832e-4016-a78a-ce1a619ad829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:25.872857 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a1443-832e-4016-a78a-ce1a619ad829 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:25.873140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a1443-832e-4016-a78a-ce1a619ad829 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:25.873444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a1443-832e-4016-a78a-ce1a619ad829 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:25.874198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a1443-832e-4016-a78a-ce1a619ad829 None admin] RBAC: Adding query filter params (name=member) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:25.874508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a1443-832e-4016-a78a-ce1a619ad829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:25.875881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a1443-832e-4016-a78a-ce1a619ad829 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:25.885352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 166/332] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:25.892291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee19c0-13ec-42a1-9a4d-c2e9c310c54f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:25.988065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:26.048888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee19c0-13ec-42a1-9a4d-c2e9c310c54f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:26.094081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502e35f-4ca0-49d4-a3e5-8ba5f717bd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:26.094971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502e35f-4ca0-49d4-a3e5-8ba5f717bd4b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:26.095258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502e35f-4ca0-49d4-a3e5-8ba5f717bd4b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:26.095533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502e35f-4ca0-49d4-a3e5-8ba5f717bd4b None admin] PATH_INFO: `/v3/users/d75c0c9bb2b941829a30b68cbdf557be` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:26.096752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502e35f-4ca0-49d4-a3e5-8ba5f717bd4b None admin] RBAC: Authorizing `identity:get_user(user_id=d75c0c9bb2b941829a30b68cbdf557be)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:26.098242 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502e35f-4ca0-49d4-a3e5-8ba5f717bd4b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:26.100186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 167/333] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:25 2022] GET /identity/v3/users/d75c0c9bb2b941829a30b68cbdf557be => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:26.105899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a1443-832e-4016-a78a-ce1a619ad829 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:26.235931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:26.248293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a1443-832e-4016-a78a-ce1a619ad829 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:26.296624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ba8e9-1bb5-4006-92a6-e690691959ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:26.297414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ba8e9-1bb5-4006-92a6-e690691959ed None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:26.297633 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ba8e9-1bb5-4006-92a6-e690691959ed None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:26.297912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ba8e9-1bb5-4006-92a6-e690691959ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:26.298753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ba8e9-1bb5-4006-92a6-e690691959ed None admin] RBAC: Adding query filter params (role.id=0960434fd9da404ea96841eb4306eaae, user.id=d75c0c9bb2b941829a30b68cbdf557be, scope.system=all) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:26.299072 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ba8e9-1bb5-4006-92a6-e690691959ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:26.300585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ba8e9-1bb5-4006-92a6-e690691959ed None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:26.314325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 167/334] 10.222.0.81 () {56 vars in 1308 bytes} [Thu Jun 16 18:37:26 2022] GET /identity/v3/role_assignments?user.id=d75c0c9bb2b941829a30b68cbdf557be&scope.system=all&role.id=0960434fd9da404ea96841eb4306eaae => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:27.767185 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148f343-7e50-4fdf-aaf6-d453fcf09777 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:27.768164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148f343-7e50-4fdf-aaf6-d453fcf09777 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:27.768660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148f343-7e50-4fdf-aaf6-d453fcf09777 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:27.770639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 168/335] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:27.776214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160de83-946d-455b-80e1-a76d94d45e07 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:27.777062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160de83-946d-455b-80e1-a76d94d45e07 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:27.777588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160de83-946d-455b-80e1-a76d94d45e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:27.802244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-a160de83-946d-455b-80e1-a76d94d45e07 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:27.803781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a160de83-946d-455b-80e1-a76d94d45e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:27.897212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 168/336] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:27.934192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148f343-7e50-4fdf-aaf6-d453fcf09777 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:28.048849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a892cdf-58bc-404b-a7a5-63a9bcaec653 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:28.049499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a892cdf-58bc-404b-a7a5-63a9bcaec653 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:28.049979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a892cdf-58bc-404b-a7a5-63a9bcaec653 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:28.051541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 169/337] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:37:28.056890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520c7c8-61e4-4323-86f9-1a33519cadb5 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:28.057182 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520c7c8-61e4-4323-86f9-1a33519cadb5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:28.057507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520c7c8-61e4-4323-86f9-1a33519cadb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:28.062518 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148f343-7e50-4fdf-aaf6-d453fcf09777 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:28.090296 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-7520c7c8-61e4-4323-86f9-1a33519cadb5 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:28.091480 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7520c7c8-61e4-4323-86f9-1a33519cadb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:28.097596 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:28.109754 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fe2ac-378d-4df3-a505-375c08dcf031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:28.110586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fe2ac-378d-4df3-a505-375c08dcf031 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:28.110860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fe2ac-378d-4df3-a505-375c08dcf031 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:28.111139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fe2ac-378d-4df3-a505-375c08dcf031 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:28.130149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fe2ac-378d-4df3-a505-375c08dcf031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-569fe2ac-378d-4df3-a505-375c08dcf031 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:28.139022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:28.139952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:28.141558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 169/338] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:28.146234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fe2ac-378d-4df3-a505-375c08dcf031 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:28.207079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 170/339] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:28.248062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520c7c8-61e4-4323-86f9-1a33519cadb5 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:28.275160 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:28.308938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fe2ac-378d-4df3-a505-375c08dcf031 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:28.378774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033e94a-8d43-4ef2-8369-bca31bfde25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:28.379455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033e94a-8d43-4ef2-8369-bca31bfde25d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:28.379742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033e94a-8d43-4ef2-8369-bca31bfde25d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:28.380021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033e94a-8d43-4ef2-8369-bca31bfde25d None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:28.380771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033e94a-8d43-4ef2-8369-bca31bfde25d None admin] RBAC: Adding query filter params (name=member) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:28.381090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033e94a-8d43-4ef2-8369-bca31bfde25d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:28.382666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033e94a-8d43-4ef2-8369-bca31bfde25d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:28.393939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 170/340] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:28.399526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033e94a-8d43-4ef2-8369-bca31bfde25d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:28.441162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520c7c8-61e4-4323-86f9-1a33519cadb5 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:28.503369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:28.508795 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c7ba1-9bf1-4870-9c09-4f9e7675287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:28.509638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c7ba1-9bf1-4870-9c09-4f9e7675287b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:28.509920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c7ba1-9bf1-4870-9c09-4f9e7675287b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:28.510190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c7ba1-9bf1-4870-9c09-4f9e7675287b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:28.531725 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c7ba1-9bf1-4870-9c09-4f9e7675287b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-189c7ba1-9bf1-4870-9c09-4f9e7675287b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:28.543667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:28.544956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:28.545789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:28.553380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 171/341] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:37:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:28.556481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c7ba1-9bf1-4870-9c09-4f9e7675287b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:28.587313 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033e94a-8d43-4ef2-8369-bca31bfde25d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:28.634074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bec5bb-93b8-4754-97a3-647bab12b72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:28.635450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bec5bb-93b8-4754-97a3-647bab12b72b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:28.637351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bec5bb-93b8-4754-97a3-647bab12b72b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:28.637845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bec5bb-93b8-4754-97a3-647bab12b72b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:28.705674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:28.739587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bec5bb-93b8-4754-97a3-647bab12b72b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:28.741423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bec5bb-93b8-4754-97a3-647bab12b72b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:28.783373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c7ba1-9bf1-4870-9c09-4f9e7675287b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:28.795861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:28.854604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 171/342] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:28.856889 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7c784-cdf3-43ce-a074-9399829b520c None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:28.857555 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7c784-cdf3-43ce-a074-9399829b520c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:28.858576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7c784-cdf3-43ce-a074-9399829b520c None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:28.860451 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 172/343] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:28.861911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7c784-cdf3-43ce-a074-9399829b520c None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:28.868985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb61914-e693-4ea0-9638-f04a2146eb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:28.870006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb61914-e693-4ea0-9638-f04a2146eb55 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:28.870497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb61914-e693-4ea0-9638-f04a2146eb55 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:28.870960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb61914-e693-4ea0-9638-f04a2146eb55 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:28.871887 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb61914-e693-4ea0-9638-f04a2146eb55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:28.872376 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb61914-e693-4ea0-9638-f04a2146eb55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:28.873974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb61914-e693-4ea0-9638-f04a2146eb55 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:28.897322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 172/344] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:28.899356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbfdce-5f4f-4727-b2da-fa34a5be33bb None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:28.899604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbfdce-5f4f-4727-b2da-fa34a5be33bb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:28.899900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbfdce-5f4f-4727-b2da-fa34a5be33bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:28.931374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4ecbfdce-5f4f-4727-b2da-fa34a5be33bb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:28.932549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4ecbfdce-5f4f-4727-b2da-fa34a5be33bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:28.996346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7c784-cdf3-43ce-a074-9399829b520c None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:29.037545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79459310-bb76-4b85-9d8f-e6523bc0d353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:29.038804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79459310-bb76-4b85-9d8f-e6523bc0d353 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:29.039295 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79459310-bb76-4b85-9d8f-e6523bc0d353 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:29.039788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79459310-bb76-4b85-9d8f-e6523bc0d353 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:29.041477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79459310-bb76-4b85-9d8f-e6523bc0d353 None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:29.043218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79459310-bb76-4b85-9d8f-e6523bc0d353 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:29.045470 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 173/345] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:28 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:29.046504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79459310-bb76-4b85-9d8f-e6523bc0d353 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:29.051361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 173/346] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:29.052023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457abbf-d3cf-43f4-ae9c-76da9fd9f475 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:29.052314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457abbf-d3cf-43f4-ae9c-76da9fd9f475 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:29.052591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457abbf-d3cf-43f4-ae9c-76da9fd9f475 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:29.053878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 174/347] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:29.057098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457abbf-d3cf-43f4-ae9c-76da9fd9f475 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:29.077123 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:29.150868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:29.173553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79459310-bb76-4b85-9d8f-e6523bc0d353 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:29.197007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457abbf-d3cf-43f4-ae9c-76da9fd9f475 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:29.212632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f54f6-33e0-4a3d-97f1-214293423631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:29.213426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f54f6-33e0-4a3d-97f1-214293423631 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:29.213716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f54f6-33e0-4a3d-97f1-214293423631 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:29.213991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f54f6-33e0-4a3d-97f1-214293423631 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:29.245226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908fbca0-c964-42af-8b31-a3a2cf1760f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:29.246479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fbca0-c964-42af-8b31-a3a2cf1760f8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:29.247029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fbca0-c964-42af-8b31-a3a2cf1760f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:29.247311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908fbca0-c964-42af-8b31-a3a2cf1760f8 None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:29.255181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fbca0-c964-42af-8b31-a3a2cf1760f8 None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:29.256793 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908fbca0-c964-42af-8b31-a3a2cf1760f8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:29.260220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 175/348] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:29 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:29.261458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ef429-4945-4651-979f-229bb8e29772 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:29.261743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ef429-4945-4651-979f-229bb8e29772 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:29.262034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ef429-4945-4651-979f-229bb8e29772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:29.284451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-198ef429-4945-4651-979f-229bb8e29772 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:29.285674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-198ef429-4945-4651-979f-229bb8e29772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:29.304106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f54f6-33e0-4a3d-97f1-214293423631 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:29.306490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f54f6-33e0-4a3d-97f1-214293423631 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:29.330507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:29.386586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 176/349] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:29.387743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ef429-4945-4651-979f-229bb8e29772 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:29.438565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 174/350] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:29.439908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa7545-1581-48a1-b321-e013e65f123a None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:29.440481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa7545-1581-48a1-b321-e013e65f123a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:29.440599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa7545-1581-48a1-b321-e013e65f123a None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:29.442185 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 175/351] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:29.442902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa7545-1581-48a1-b321-e013e65f123a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:29.464235 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:29.521264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ef429-4945-4651-979f-229bb8e29772 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:29.559981 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67626a-ee2c-4507-a74e-7a26443cf9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:29.561360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67626a-ee2c-4507-a74e-7a26443cf9d8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:29.561781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67626a-ee2c-4507-a74e-7a26443cf9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:29.562059 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67626a-ee2c-4507-a74e-7a26443cf9d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:29.564113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa7545-1581-48a1-b321-e013e65f123a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:29.576788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67626a-ee2c-4507-a74e-7a26443cf9d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-9d67626a-ee2c-4507-a74e-7a26443cf9d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:29.585455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:29.586403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:29.587213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:29.587841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:29.588909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 177/352] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:29.592085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67626a-ee2c-4507-a74e-7a26443cf9d8 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:29.607062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f0213-b91f-448b-83db-ad408a406719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:29.607721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f0213-b91f-448b-83db-ad408a406719 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:29.608003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f0213-b91f-448b-83db-ad408a406719 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:29.608276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f0213-b91f-448b-83db-ad408a406719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:29.609728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f0213-b91f-448b-83db-ad408a406719 None admin] RBAC: Adding query filter params (group.id=24f62c77801c46a5a9271f50cbac1161, role.id=0960434fd9da404ea96841eb4306eaae, scope.project.id=0e628770474b423e99809dd568e34a0e) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:29.609852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f0213-b91f-448b-83db-ad408a406719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:29.611561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f0213-b91f-448b-83db-ad408a406719 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:29.620050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 176/353] 10.222.0.81 () {56 vars in 1376 bytes} [Thu Jun 16 18:37:29 2022] GET /identity/v3/role_assignments?group.id=24f62c77801c46a5a9271f50cbac1161&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=0960434fd9da404ea96841eb4306eaae => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:29.621110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ac2c3-3998-4d39-b2a2-3a2f86beb0b8 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:29.626921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ac2c3-3998-4d39-b2a2-3a2f86beb0b8 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:29.627130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31ac2c3-3998-4d39-b2a2-3a2f86beb0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:29.649519 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f31ac2c3-3998-4d39-b2a2-3a2f86beb0b8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:29.651340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f31ac2c3-3998-4d39-b2a2-3a2f86beb0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:29.755236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 177/354] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:29.755914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ac2c3-3998-4d39-b2a2-3a2f86beb0b8 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:29.769153 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67626a-ee2c-4507-a74e-7a26443cf9d8 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:29.795811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:29.816340 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d90cc4-2b1f-4bbf-9d1d-0f0d868b6937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:29.817272 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d90cc4-2b1f-4bbf-9d1d-0f0d868b6937 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:29.817593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d90cc4-2b1f-4bbf-9d1d-0f0d868b6937 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:29.817879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d90cc4-2b1f-4bbf-9d1d-0f0d868b6937 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:29.835378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d90cc4-2b1f-4bbf-9d1d-0f0d868b6937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-27d90cc4-2b1f-4bbf-9d1d-0f0d868b6937 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:29.846585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:29.847760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:29.848743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 178/355] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:37:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:29.854253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d90cc4-2b1f-4bbf-9d1d-0f0d868b6937 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:29.900685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31ac2c3-3998-4d39-b2a2-3a2f86beb0b8 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:29.913148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:29.943423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f609b14-d74f-4d86-b296-779b6c88868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:29.944866 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f609b14-d74f-4d86-b296-779b6c88868d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:29.949059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f609b14-d74f-4d86-b296-779b6c88868d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:29.949059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f609b14-d74f-4d86-b296-779b6c88868d None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:29.950089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f609b14-d74f-4d86-b296-779b6c88868d None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:29.951576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f609b14-d74f-4d86-b296-779b6c88868d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:29.953728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 178/356] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:29 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:29.955021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f609b14-d74f-4d86-b296-779b6c88868d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:29.980868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d90cc4-2b1f-4bbf-9d1d-0f0d868b6937 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:30.020162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a8d8d-d8b3-459a-a043-7f9b2c09dfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:30.020763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a8d8d-d8b3-459a-a043-7f9b2c09dfec None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:30.021051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a8d8d-d8b3-459a-a043-7f9b2c09dfec None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:30.021361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a8d8d-d8b3-459a-a043-7f9b2c09dfec None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:30.022117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a8d8d-d8b3-459a-a043-7f9b2c09dfec None admin] RBAC: Adding query filter params (name=member) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:30.022430 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a8d8d-d8b3-459a-a043-7f9b2c09dfec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:30.024524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a8d8d-d8b3-459a-a043-7f9b2c09dfec None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:30.035404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 179/357] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:30.036399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a8d8d-d8b3-459a-a043-7f9b2c09dfec None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:30.090197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f609b14-d74f-4d86-b296-779b6c88868d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:30.090492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:30.138240 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a67f0-b199-47a2-8ba2-e52bc681fadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:30.138845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a67f0-b199-47a2-8ba2-e52bc681fadc None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:30.139126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a67f0-b199-47a2-8ba2-e52bc681fadc None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:30.139560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a67f0-b199-47a2-8ba2-e52bc681fadc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:30.152957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a67f0-b199-47a2-8ba2-e52bc681fadc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-ce0a67f0-b199-47a2-8ba2-e52bc681fadc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:30.160774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:30.162044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 18:37:30.163069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:30.163806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 179/358] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:30.164436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45fb5-af02-4377-990e-123b20558bac None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:30.164739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45fb5-af02-4377-990e-123b20558bac None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:30.165003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc45fb5-af02-4377-990e-123b20558bac None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:30.166494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 180/359] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:30.167435 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc45fb5-af02-4377-990e-123b20558bac None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:30.175839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a8d8d-d8b3-459a-a043-7f9b2c09dfec None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:30.216330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72512184-bccf-475e-a12a-02a5ac150f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:30.217007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72512184-bccf-475e-a12a-02a5ac150f9e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:30.217383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72512184-bccf-475e-a12a-02a5ac150f9e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:30.217611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72512184-bccf-475e-a12a-02a5ac150f9e None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:30.218409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72512184-bccf-475e-a12a-02a5ac150f9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:30.218722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72512184-bccf-475e-a12a-02a5ac150f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:30.220148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72512184-bccf-475e-a12a-02a5ac150f9e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:30.221098 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:30.231069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 180/360] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:30.231803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72512184-bccf-475e-a12a-02a5ac150f9e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:30.306748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc45fb5-af02-4377-990e-123b20558bac None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:30.352780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906791f-6692-45d8-8daf-32da671993b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:30.353677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906791f-6692-45d8-8daf-32da671993b9 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:30.353965 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906791f-6692-45d8-8daf-32da671993b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:30.354400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906791f-6692-45d8-8daf-32da671993b9 None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:30.355462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906791f-6692-45d8-8daf-32da671993b9 None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:30.356845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72512184-bccf-475e-a12a-02a5ac150f9e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:30.357012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906791f-6692-45d8-8daf-32da671993b9 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:30.361623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 181/361] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:30 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:30.361623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906791f-6692-45d8-8daf-32da671993b9 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:30.365284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:30.403649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c228e06-1bb4-4d49-8b09-8c028cc6a6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:30.404296 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c228e06-1bb4-4d49-8b09-8c028cc6a6c4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:30.404577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c228e06-1bb4-4d49-8b09-8c028cc6a6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:30.404850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c228e06-1bb4-4d49-8b09-8c028cc6a6c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:30.486731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c228e06-1bb4-4d49-8b09-8c028cc6a6c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:30.488530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c228e06-1bb4-4d49-8b09-8c028cc6a6c4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:30.489918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906791f-6692-45d8-8daf-32da671993b9 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:30.499397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:30.543485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13588c91-8699-4e24-9c36-f057695c7f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:30.544186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13588c91-8699-4e24-9c36-f057695c7f7d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:30.544473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13588c91-8699-4e24-9c36-f057695c7f7d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:30.544749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13588c91-8699-4e24-9c36-f057695c7f7d None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:30.545519 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13588c91-8699-4e24-9c36-f057695c7f7d None admin] RBAC: Adding query filter params (name=reader) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:30.545828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13588c91-8699-4e24-9c36-f057695c7f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:30.547210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13588c91-8699-4e24-9c36-f057695c7f7d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:30.556931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 182/362] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:30.557947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9bd46-f150-43e4-8b04-4b7c93d4cf63 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:30.558234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9bd46-f150-43e4-8b04-4b7c93d4cf63 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:30.558514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd9bd46-f150-43e4-8b04-4b7c93d4cf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:30.577453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-4fd9bd46-f150-43e4-8b04-4b7c93d4cf63 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:30.578672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-4fd9bd46-f150-43e4-8b04-4b7c93d4cf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:30.609730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 181/363] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:30.610414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c228e06-1bb4-4d49-8b09-8c028cc6a6c4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:30.615152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:30.669001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 183/364] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:30.670083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9bd46-f150-43e4-8b04-4b7c93d4cf63 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:30.738520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c228e06-1bb4-4d49-8b09-8c028cc6a6c4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:30.777065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e7040-15aa-4eed-b91e-d6960b0a2a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:30.777753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e7040-15aa-4eed-b91e-d6960b0a2a50 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:30.778039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e7040-15aa-4eed-b91e-d6960b0a2a50 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:30.778316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e7040-15aa-4eed-b91e-d6960b0a2a50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:30.785194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:30.791962 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd9bd46-f150-43e4-8b04-4b7c93d4cf63 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:30.836488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cb930-c72a-4e67-93ed-d1277debcb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:30.837084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cb930-c72a-4e67-93ed-d1277debcb4c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:30.837401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cb930-c72a-4e67-93ed-d1277debcb4c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:30.838075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cb930-c72a-4e67-93ed-d1277debcb4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:30.839008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251cb930-c72a-4e67-93ed-d1277debcb4c None admin] RBAC: Adding query filter params (group.id=24f62c77801c46a5a9271f50cbac1161, role.id=5f24daf2ba2b4f8fab95c87cefb52231, scope.project.id=0e628770474b423e99809dd568e34a0e) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:30.839327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251cb930-c72a-4e67-93ed-d1277debcb4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:30.841216 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251cb930-c72a-4e67-93ed-d1277debcb4c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:30.847279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 184/365] 10.222.0.81 () {56 vars in 1376 bytes} [Thu Jun 16 18:37:30 2022] GET /identity/v3/role_assignments?group.id=24f62c77801c46a5a9271f50cbac1161&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=5f24daf2ba2b4f8fab95c87cefb52231 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:30.848063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cb930-c72a-4e67-93ed-d1277debcb4c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:30.872350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e7040-15aa-4eed-b91e-d6960b0a2a50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:30.873916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e7040-15aa-4eed-b91e-d6960b0a2a50 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:30.879957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:30.986625 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cb930-c72a-4e67-93ed-d1277debcb4c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:31.028597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 182/366] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:31.029956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e7040-15aa-4eed-b91e-d6960b0a2a50 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:31.030081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0667696-4653-4c3d-97b6-a604770b57b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:31.030908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667696-4653-4c3d-97b6-a604770b57b7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:31.031198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667696-4653-4c3d-97b6-a604770b57b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:31.031471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0667696-4653-4c3d-97b6-a604770b57b7 None admin] PATH_INFO: `/v3/users/da63b34c457e44b1834608951ba16839` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:31.032699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0667696-4653-4c3d-97b6-a604770b57b7 None admin] RBAC: Authorizing `identity:get_user(user_id=da63b34c457e44b1834608951ba16839)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:31.034457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0667696-4653-4c3d-97b6-a604770b57b7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:31.036129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 185/367] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:37:30 2022] GET /identity/v3/users/da63b34c457e44b1834608951ba16839 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:31.036905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0667696-4653-4c3d-97b6-a604770b57b7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:31.077937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:31.155188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0667696-4653-4c3d-97b6-a604770b57b7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:31.163805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e7040-15aa-4eed-b91e-d6960b0a2a50 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:31.167847 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:31.197605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf67e4-b099-4161-afe7-36cd1b21887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:31.198406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf67e4-b099-4161-afe7-36cd1b21887c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:31.198693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf67e4-b099-4161-afe7-36cd1b21887c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:31.198965 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf67e4-b099-4161-afe7-36cd1b21887c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:31.216556 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf67e4-b099-4161-afe7-36cd1b21887c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:31.221429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326db0e2-7917-4074-9b25-e5c11d3e6350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:31.222301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326db0e2-7917-4074-9b25-e5c11d3e6350 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:31.222585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326db0e2-7917-4074-9b25-e5c11d3e6350 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:31.222868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326db0e2-7917-4074-9b25-e5c11d3e6350 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:31.224053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326db0e2-7917-4074-9b25-e5c11d3e6350 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:31.225730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326db0e2-7917-4074-9b25-e5c11d3e6350 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:31.227980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 183/368] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:31 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-c6bf67e4-b099-4161-afe7-36cd1b21887c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:31.228717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:31.229650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:31.230879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:31.231524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326db0e2-7917-4074-9b25-e5c11d3e6350 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:31.237636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 186/369] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:37:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:31.238678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf67e4-b099-4161-afe7-36cd1b21887c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:31.337152 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:31.349274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf67e4-b099-4161-afe7-36cd1b21887c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:31.349783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326db0e2-7917-4074-9b25-e5c11d3e6350 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:31.389752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e9a06-08bc-40c7-9c84-ff6708d1cb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:31.390699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cd570-2e2f-4db3-b0fd-8ac95fbb6ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:31.391128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e9a06-08bc-40c7-9c84-ff6708d1cb19 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:31.391553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cd570-2e2f-4db3-b0fd-8ac95fbb6ce7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:31.391801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e9a06-08bc-40c7-9c84-ff6708d1cb19 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:31.392052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cd570-2e2f-4db3-b0fd-8ac95fbb6ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:31.392365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e9a06-08bc-40c7-9c84-ff6708d1cb19 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:31.392477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cd570-2e2f-4db3-b0fd-8ac95fbb6ce7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:31.393450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cd570-2e2f-4db3-b0fd-8ac95fbb6ce7 None admin] RBAC: Adding query filter params (role.id=43a16638caac4dd8acae2987c010b48a, user.id=da63b34c457e44b1834608951ba16839, scope.system=all) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:31.393863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cd570-2e2f-4db3-b0fd-8ac95fbb6ce7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:31.394138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e9a06-08bc-40c7-9c84-ff6708d1cb19 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:31.395507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cd570-2e2f-4db3-b0fd-8ac95fbb6ce7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:31.395958 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e9a06-08bc-40c7-9c84-ff6708d1cb19 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:31.398335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 184/370] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:31 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:31.399935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e9a06-08bc-40c7-9c84-ff6708d1cb19 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:31.413279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 187/371] 10.222.0.81 () {56 vars in 1308 bytes} [Thu Jun 16 18:37:31 2022] GET /identity/v3/role_assignments?user.id=da63b34c457e44b1834608951ba16839&scope.system=all&role.id=43a16638caac4dd8acae2987c010b48a => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:31.414211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cd570-2e2f-4db3-b0fd-8ac95fbb6ce7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:31.450612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:31.520208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e9a06-08bc-40c7-9c84-ff6708d1cb19 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:31.527285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cd570-2e2f-4db3-b0fd-8ac95fbb6ce7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:31.561477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7806cb-1d93-4721-a54b-964164116546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:31.562990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7806cb-1d93-4721-a54b-964164116546 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:31.563509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7806cb-1d93-4721-a54b-964164116546 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:31.564010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7806cb-1d93-4721-a54b-964164116546 None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:31.565807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7806cb-1d93-4721-a54b-964164116546 None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:31.566377 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27584f-93ed-4ec1-9c46-4ff1091b1353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:31.567160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27584f-93ed-4ec1-9c46-4ff1091b1353 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:31.567546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27584f-93ed-4ec1-9c46-4ff1091b1353 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:31.567922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27584f-93ed-4ec1-9c46-4ff1091b1353 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:31.568103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7806cb-1d93-4721-a54b-964164116546 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:31.569184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27584f-93ed-4ec1-9c46-4ff1091b1353 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:31.569184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27584f-93ed-4ec1-9c46-4ff1091b1353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:31.570590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 185/372] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:31 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:31.571218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27584f-93ed-4ec1-9c46-4ff1091b1353 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:31.571954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7806cb-1d93-4721-a54b-964164116546 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:31.590029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 188/373] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:31.590029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27584f-93ed-4ec1-9c46-4ff1091b1353 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:31.605886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:31.712184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7806cb-1d93-4721-a54b-964164116546 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:31.714577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27584f-93ed-4ec1-9c46-4ff1091b1353 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:31.725729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:31.759455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547dde2-7e14-48e5-a4ae-4f6e2d873e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:31.760319 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547dde2-7e14-48e5-a4ae-4f6e2d873e12 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:31.760605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547dde2-7e14-48e5-a4ae-4f6e2d873e12 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:31.760878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547dde2-7e14-48e5-a4ae-4f6e2d873e12 None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:31.761542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef466875-6e92-45d4-b7fc-9dd903d72a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:31.762150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547dde2-7e14-48e5-a4ae-4f6e2d873e12 None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:31.762254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef466875-6e92-45d4-b7fc-9dd903d72a96 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:31.762479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef466875-6e92-45d4-b7fc-9dd903d72a96 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:31.762763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef466875-6e92-45d4-b7fc-9dd903d72a96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:31.763581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef466875-6e92-45d4-b7fc-9dd903d72a96 None admin] RBAC: Adding query filter params (group.id=24f62c77801c46a5a9271f50cbac1161, role.id=0960434fd9da404ea96841eb4306eaae, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:31.763680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9547dde2-7e14-48e5-a4ae-4f6e2d873e12 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:31.763895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef466875-6e92-45d4-b7fc-9dd903d72a96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:31.765284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 186/374] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:31 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:31.765647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef466875-6e92-45d4-b7fc-9dd903d72a96 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:31.766047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547dde2-7e14-48e5-a4ae-4f6e2d873e12 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:31.777369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 189/375] 10.222.0.81 () {56 vars in 1376 bytes} [Thu Jun 16 18:37:31 2022] GET /identity/v3/role_assignments?group.id=24f62c77801c46a5a9271f50cbac1161&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=0960434fd9da404ea96841eb4306eaae => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:31.778292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef466875-6e92-45d4-b7fc-9dd903d72a96 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:31.879681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:31.908996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547dde2-7e14-48e5-a4ae-4f6e2d873e12 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:31.911989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef466875-6e92-45d4-b7fc-9dd903d72a96 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:31.955156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245b7df-57ee-422f-a51a-c7b1bf7db005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:31.955825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245b7df-57ee-422f-a51a-c7b1bf7db005 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:31.956079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb025e-bffc-4374-adfd-4380d37c8a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:31.956213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245b7df-57ee-422f-a51a-c7b1bf7db005 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:31.956394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245b7df-57ee-422f-a51a-c7b1bf7db005 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:31.956737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb025e-bffc-4374-adfd-4380d37c8a2b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:31.957025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb025e-bffc-4374-adfd-4380d37c8a2b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:31.957190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245b7df-57ee-422f-a51a-c7b1bf7db005 None admin] RBAC: Adding query filter params (group.id=24f62c77801c46a5a9271f50cbac1161, role.id=5f24daf2ba2b4f8fab95c87cefb52231, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:31.957318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb025e-bffc-4374-adfd-4380d37c8a2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:31.959531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245b7df-57ee-422f-a51a-c7b1bf7db005 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:31.961201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245b7df-57ee-422f-a51a-c7b1bf7db005 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:31.970670 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 190/376] 10.222.0.81 () {56 vars in 1376 bytes} [Thu Jun 16 18:37:31 2022] GET /identity/v3/role_assignments?group.id=24f62c77801c46a5a9271f50cbac1161&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=5f24daf2ba2b4f8fab95c87cefb52231 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:32.032589 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:32.068509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb025e-bffc-4374-adfd-4380d37c8a2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:32.070169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb025e-bffc-4374-adfd-4380d37c8a2b None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:32.210191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 187/377] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:32.215777 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245b7df-57ee-422f-a51a-c7b1bf7db005 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:32.298421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b2299-8277-43fd-9c64-785fcfa0ef5a None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:32.299031 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b2299-8277-43fd-9c64-785fcfa0ef5a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:32.299521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70b2299-8277-43fd-9c64-785fcfa0ef5a None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:32.301376 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 188/378] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:37:32.307378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b597e-8c3f-4669-90d8-c7ab38983d37 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:32.307954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b597e-8c3f-4669-90d8-c7ab38983d37 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:32.308436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b597e-8c3f-4669-90d8-c7ab38983d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:32.327392 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-560b597e-8c3f-4669-90d8-c7ab38983d37 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:32.328894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-560b597e-8c3f-4669-90d8-c7ab38983d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:32.379858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245b7df-57ee-422f-a51a-c7b1bf7db005 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:32.442651 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e3d64-eeae-49f0-8323-227da18f5e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:32.444092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e3d64-eeae-49f0-8323-227da18f5e1a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:32.445390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e3d64-eeae-49f0-8323-227da18f5e1a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:32.447078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e3d64-eeae-49f0-8323-227da18f5e1a None admin] PATH_INFO: `/v3/projects/3b167122bd2b4c3cba143eecf3576355` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:32.450039 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e3d64-eeae-49f0-8323-227da18f5e1a None admin] RBAC: Authorizing `identity:get_project(project_id=3b167122bd2b4c3cba143eecf3576355)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:32.451733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e3d64-eeae-49f0-8323-227da18f5e1a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:32.461213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 191/380] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:32 2022] GET /identity/v3/projects/3b167122bd2b4c3cba143eecf3576355 => generated 344 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:32.461747 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 189/380] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:32.468799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e3d64-eeae-49f0-8323-227da18f5e1a None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:32.484044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:32.486930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b597e-8c3f-4669-90d8-c7ab38983d37 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:32.498050 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:32.618645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b597e-8c3f-4669-90d8-c7ab38983d37 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:32.640998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e3d64-eeae-49f0-8323-227da18f5e1a None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:32.673964 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c031bd4-28b6-41a3-8124-3bcb4a0c4ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:32.674751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c031bd4-28b6-41a3-8124-3bcb4a0c4ccd None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:32.675039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c031bd4-28b6-41a3-8124-3bcb4a0c4ccd None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:32.675317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c031bd4-28b6-41a3-8124-3bcb4a0c4ccd None admin] PATH_INFO: `/v3/roles/member` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:32.682189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbddb2-6372-438f-9d3d-197f0992dff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:32.683292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbddb2-6372-438f-9d3d-197f0992dff9 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:32.683823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbddb2-6372-438f-9d3d-197f0992dff9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:32.684299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbddb2-6372-438f-9d3d-197f0992dff9 None admin] PATH_INFO: `/v3/groups/2dd9d6313e5a49e9a89359abfa2ba378` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:32.690262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c031bd4-28b6-41a3-8124-3bcb4a0c4ccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:32.690760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbddb2-6372-438f-9d3d-197f0992dff9 None admin] RBAC: Authorizing `identity:get_group(group_id=2dd9d6313e5a49e9a89359abfa2ba378)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:32.692420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbddb2-6372-438f-9d3d-197f0992dff9 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:32.699189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 192/381] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:32 2022] GET /identity/v3/groups/2dd9d6313e5a49e9a89359abfa2ba378 => generated 229 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:32.700640 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbddb2-6372-438f-9d3d-197f0992dff9 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-3c031bd4-28b6-41a3-8124-3bcb4a0c4ccd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:32.702206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:32.703053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:32.704229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 190/382] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:32.710177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c031bd4-28b6-41a3-8124-3bcb4a0c4ccd None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:32.822059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:32.839546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:32.879673 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbddb2-6372-438f-9d3d-197f0992dff9 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:32.912504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c031bd4-28b6-41a3-8124-3bcb4a0c4ccd None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:32.945497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b11e07-cf65-49f1-a7d6-df5713f486f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:32.946195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b11e07-cf65-49f1-a7d6-df5713f486f8 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:32.946477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b11e07-cf65-49f1-a7d6-df5713f486f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:32.946751 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b11e07-cf65-49f1-a7d6-df5713f486f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:32.947579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b11e07-cf65-49f1-a7d6-df5713f486f8 None admin] RBAC: Adding query filter params (group.id=2dd9d6313e5a49e9a89359abfa2ba378, role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=3b167122bd2b4c3cba143eecf3576355) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:32.947889 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b11e07-cf65-49f1-a7d6-df5713f486f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:32.952112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b11e07-cf65-49f1-a7d6-df5713f486f8 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:32.964636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 193/383] 10.222.0.81 () {56 vars in 1376 bytes} [Thu Jun 16 18:37:32 2022] GET /identity/v3/role_assignments?group.id=2dd9d6313e5a49e9a89359abfa2ba378&scope.project.id=3b167122bd2b4c3cba143eecf3576355&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:32.971439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c36d58-e602-411c-99bb-b72167dc50c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:32.972127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c36d58-e602-411c-99bb-b72167dc50c2 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:32.972410 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c36d58-e602-411c-99bb-b72167dc50c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:32.972685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c36d58-e602-411c-99bb-b72167dc50c2 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:32.973996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c36d58-e602-411c-99bb-b72167dc50c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:32.974363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c36d58-e602-411c-99bb-b72167dc50c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:32.975776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c36d58-e602-411c-99bb-b72167dc50c2 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:32.986403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 191/384] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:32.991827 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b11e07-cf65-49f1-a7d6-df5713f486f8 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:33.104443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b11e07-cf65-49f1-a7d6-df5713f486f8 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:33.139848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eab10-9aac-40a5-9214-307a63e754f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:33.141015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eab10-9aac-40a5-9214-307a63e754f3 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:33.141505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eab10-9aac-40a5-9214-307a63e754f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:33.141952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eab10-9aac-40a5-9214-307a63e754f3 None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:33.143436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eab10-9aac-40a5-9214-307a63e754f3 None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:33.145052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eab10-9aac-40a5-9214-307a63e754f3 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:33.147220 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 194/385] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:32 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:33.153004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c36d58-e602-411c-99bb-b72167dc50c2 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:33.312424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c36d58-e602-411c-99bb-b72167dc50c2 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:33.347110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:33.361975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc7b71-4692-4d74-934d-cecc751c34ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:33.362860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc7b71-4692-4d74-934d-cecc751c34ed None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:33.363147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc7b71-4692-4d74-934d-cecc751c34ed None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:33.363431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc7b71-4692-4d74-934d-cecc751c34ed None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:33.365494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc7b71-4692-4d74-934d-cecc751c34ed None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:33.377600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc7b71-4692-4d74-934d-cecc751c34ed None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:33.377600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 192/386] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:33 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:33.378310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eab10-9aac-40a5-9214-307a63e754f3 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:33.379119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:33.513349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eab10-9aac-40a5-9214-307a63e754f3 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:33.547874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cf86e-ae93-4e3f-954b-0480a382ca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:33.548765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cf86e-ae93-4e3f-954b-0480a382ca7e None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:33.549053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cf86e-ae93-4e3f-954b-0480a382ca7e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:33.549356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cf86e-ae93-4e3f-954b-0480a382ca7e None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e/groups/24f62c77801c46a5a9271f50cbac1161/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:33.551334 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44cf86e-ae93-4e3f-954b-0480a382ca7e None admin] RBAC: Authorizing `identity:create_grant(project_id=0e628770474b423e99809dd568e34a0e, group_id=24f62c77801c46a5a9271f50cbac1161, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:33.552804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44cf86e-ae93-4e3f-954b-0480a382ca7e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:33.649149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 195/387] 10.222.0.81 () {58 vars in 1442 bytes} [Thu Jun 16 18:37:33 2022] PUT /identity/v3/projects/0e628770474b423e99809dd568e34a0e/groups/24f62c77801c46a5a9271f50cbac1161/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:33.766762 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971f673-ea03-4916-9806-8135fed1c523 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:33.766953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971f673-ea03-4916-9806-8135fed1c523 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:33.767240 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971f673-ea03-4916-9806-8135fed1c523 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:33.769055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 193/388] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:33.774988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377bcc5-dc5c-4db5-92e2-a99e59b48c1b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:33.775366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377bcc5-dc5c-4db5-92e2-a99e59b48c1b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:33.775721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377bcc5-dc5c-4db5-92e2-a99e59b48c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:33.794039 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f377bcc5-dc5c-4db5-92e2-a99e59b48c1b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:33.795257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f377bcc5-dc5c-4db5-92e2-a99e59b48c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:33.858589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:33.869794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 196/389] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:33.893470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971f673-ea03-4916-9806-8135fed1c523 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:34.005036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971f673-ea03-4916-9806-8135fed1c523 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:34.039664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d1476-849d-41f9-ae3e-6636b395507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:34.041011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d1476-849d-41f9-ae3e-6636b395507f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:34.041324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d1476-849d-41f9-ae3e-6636b395507f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:34.041658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d1476-849d-41f9-ae3e-6636b395507f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:34.052579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d1476-849d-41f9-ae3e-6636b395507f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-024d1476-849d-41f9-ae3e-6636b395507f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:34.060920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:34.061856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:34.062661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:34.068947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 194/390] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:37:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:34.070019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377bcc5-dc5c-4db5-92e2-a99e59b48c1b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:34.203195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377bcc5-dc5c-4db5-92e2-a99e59b48c1b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:34.252353 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e4fc5-074b-40d9-a82f-1f1fc0d0102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:34.253452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e4fc5-074b-40d9-a82f-1f1fc0d0102b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:34.253962 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e4fc5-074b-40d9-a82f-1f1fc0d0102b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:34.254434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e4fc5-074b-40d9-a82f-1f1fc0d0102b None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:34.255374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e4fc5-074b-40d9-a82f-1f1fc0d0102b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:34.255893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e4fc5-074b-40d9-a82f-1f1fc0d0102b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:34.257567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e4fc5-074b-40d9-a82f-1f1fc0d0102b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:34.272037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 197/391] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:34.277999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d1476-849d-41f9-ae3e-6636b395507f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:34.379271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7bb22-cec7-444d-b573-9c2bdb8a99b8 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:34.379796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:34.380629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7bb22-cec7-444d-b573-9c2bdb8a99b8 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:34.380916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd7bb22-cec7-444d-b573-9c2bdb8a99b8 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:34.383350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 198/392] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:34.387944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601c262-0be0-4183-8b32-cbf3ec4ecd28 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:34.388232 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601c262-0be0-4183-8b32-cbf3ec4ecd28 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:34.396942 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601c262-0be0-4183-8b32-cbf3ec4ecd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:34.415313 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-e601c262-0be0-4183-8b32-cbf3ec4ecd28 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:34.416518 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e601c262-0be0-4183-8b32-cbf3ec4ecd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:34.451773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d1476-849d-41f9-ae3e-6636b395507f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:34.512953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:34.517122 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ce820-0480-4a68-97fa-f8d23fe6c2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:34.519956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ce820-0480-4a68-97fa-f8d23fe6c2e2 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:34.520248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ce820-0480-4a68-97fa-f8d23fe6c2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:34.520529 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ce820-0480-4a68-97fa-f8d23fe6c2e2 None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:34.521834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ce820-0480-4a68-97fa-f8d23fe6c2e2 None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:34.523294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ce820-0480-4a68-97fa-f8d23fe6c2e2 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:34.526779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 195/393] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:34 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:34.534413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ce820-0480-4a68-97fa-f8d23fe6c2e2 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:34.557209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 199/394] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:34.579869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601c262-0be0-4183-8b32-cbf3ec4ecd28 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:34.708976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ce820-0480-4a68-97fa-f8d23fe6c2e2 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:34.733886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601c262-0be0-4183-8b32-cbf3ec4ecd28 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:34.735004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:34.747319 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ead10-77ec-40f7-b2d0-ff6e196a4d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:34.748517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ead10-77ec-40f7-b2d0-ff6e196a4d89 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:34.749059 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ead10-77ec-40f7-b2d0-ff6e196a4d89 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:34.749530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ead10-77ec-40f7-b2d0-ff6e196a4d89 None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:34.750962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ead10-77ec-40f7-b2d0-ff6e196a4d89 None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:34.752713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ead10-77ec-40f7-b2d0-ff6e196a4d89 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:34.754984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 196/395] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:34 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:34.756444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2490847-4453-46ca-baa8-a408299ed3fb None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:34.756971 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2490847-4453-46ca-baa8-a408299ed3fb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:34.757504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2490847-4453-46ca-baa8-a408299ed3fb None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:34.759243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 197/396] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:34.760274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2490847-4453-46ca-baa8-a408299ed3fb None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:34.779165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91100623-a30a-400c-b41e-77b81c779b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:34.780325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91100623-a30a-400c-b41e-77b81c779b31 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:34.780981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91100623-a30a-400c-b41e-77b81c779b31 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:34.781550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91100623-a30a-400c-b41e-77b81c779b31 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:34.793840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91100623-a30a-400c-b41e-77b81c779b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-91100623-a30a-400c-b41e-77b81c779b31 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:34.804070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:34.804959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:34.805809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:34.808525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 200/397] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:37:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:34.810068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0388c-0bf1-4b56-bfb0-ad1a1e684a5e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:34.810573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0388c-0bf1-4b56-bfb0-ad1a1e684a5e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:34.811059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0388c-0bf1-4b56-bfb0-ad1a1e684a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:34.835141 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-09e0388c-0bf1-4b56-bfb0-ad1a1e684a5e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:34.836350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-09e0388c-0bf1-4b56-bfb0-ad1a1e684a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:34.890152 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:34.901202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2490847-4453-46ca-baa8-a408299ed3fb None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:34.919247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 201/398] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:34.920534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0388c-0bf1-4b56-bfb0-ad1a1e684a5e None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:34.956911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4eb398-5803-4910-a4cf-f38e379b1f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:34.958227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4eb398-5803-4910-a4cf-f38e379b1f1a None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:34.958743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4eb398-5803-4910-a4cf-f38e379b1f1a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:34.959313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4eb398-5803-4910-a4cf-f38e379b1f1a None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e/groups/24f62c77801c46a5a9271f50cbac1161/roles/5f24daf2ba2b4f8fab95c87cefb52231` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:34.962672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4eb398-5803-4910-a4cf-f38e379b1f1a None admin] RBAC: Authorizing `identity:create_grant(project_id=0e628770474b423e99809dd568e34a0e, group_id=24f62c77801c46a5a9271f50cbac1161, role_id=5f24daf2ba2b4f8fab95c87cefb52231)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:34.964305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4eb398-5803-4910-a4cf-f38e379b1f1a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:35.036814 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 198/399] 10.222.0.81 () {58 vars in 1442 bytes} [Thu Jun 16 18:37:34 2022] PUT /identity/v3/projects/0e628770474b423e99809dd568e34a0e/groups/24f62c77801c46a5a9271f50cbac1161/roles/5f24daf2ba2b4f8fab95c87cefb52231 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:37:35.038382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0388c-0bf1-4b56-bfb0-ad1a1e684a5e None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:35.038762 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4eb398-5803-4910-a4cf-f38e379b1f1a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:35.109378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:35.112081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d12fad-f014-4c7a-a5bc-aeb6d9a917da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:35.112826 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d12fad-f014-4c7a-a5bc-aeb6d9a917da None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:35.113228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d12fad-f014-4c7a-a5bc-aeb6d9a917da None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:35.113502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d12fad-f014-4c7a-a5bc-aeb6d9a917da None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:35.114235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d12fad-f014-4c7a-a5bc-aeb6d9a917da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:35.114602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d12fad-f014-4c7a-a5bc-aeb6d9a917da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:35.116283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d12fad-f014-4c7a-a5bc-aeb6d9a917da None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:35.137008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 202/400] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:35.142863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d12fad-f014-4c7a-a5bc-aeb6d9a917da None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:35.195311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4eb398-5803-4910-a4cf-f38e379b1f1a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:35.226234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:35.245022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67d3bc-9836-4d25-b5d4-fcb02612ace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:35.246331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67d3bc-9836-4d25-b5d4-fcb02612ace0 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:35.246632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67d3bc-9836-4d25-b5d4-fcb02612ace0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:35.246982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67d3bc-9836-4d25-b5d4-fcb02612ace0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:35.262829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67d3bc-9836-4d25-b5d4-fcb02612ace0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-2e67d3bc-9836-4d25-b5d4-fcb02612ace0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:35.273411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:35.274356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:35.275143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 199/401] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:35.281869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67d3bc-9836-4d25-b5d4-fcb02612ace0 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:35.309796 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d12fad-f014-4c7a-a5bc-aeb6d9a917da None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:35.353479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b0259-acea-44f3-b11c-6e9dff6ec7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:35.354402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b0259-acea-44f3-b11c-6e9dff6ec7e1 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:35.354789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b0259-acea-44f3-b11c-6e9dff6ec7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:35.355074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4b0259-acea-44f3-b11c-6e9dff6ec7e1 None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:35.356403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b0259-acea-44f3-b11c-6e9dff6ec7e1 None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:35.358073 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4b0259-acea-44f3-b11c-6e9dff6ec7e1 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:35.369457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 203/402] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:35 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:35.371503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b0259-acea-44f3-b11c-6e9dff6ec7e1 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:35.420605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67d3bc-9836-4d25-b5d4-fcb02612ace0 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:35.434628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:35.457121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f7033-e339-425e-acdb-048511349315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:35.458161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f7033-e339-425e-acdb-048511349315 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:35.458654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f7033-e339-425e-acdb-048511349315 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:35.460022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f7033-e339-425e-acdb-048511349315 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:35.460972 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f7033-e339-425e-acdb-048511349315 None admin] RBAC: Adding query filter params (name=member) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:35.461512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f7033-e339-425e-acdb-048511349315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:35.463247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f7033-e339-425e-acdb-048511349315 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:35.472378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 200/403] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:35.478258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f7033-e339-425e-acdb-048511349315 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:35.591462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4b0259-acea-44f3-b11c-6e9dff6ec7e1 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:35.597125 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:35.660593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f7033-e339-425e-acdb-048511349315 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:35.664583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01167398-f459-4a73-adce-a2a57f5bd87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:35.666022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01167398-f459-4a73-adce-a2a57f5bd87b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:35.666307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01167398-f459-4a73-adce-a2a57f5bd87b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:35.666582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01167398-f459-4a73-adce-a2a57f5bd87b None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:35.667782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01167398-f459-4a73-adce-a2a57f5bd87b None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:35.669409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01167398-f459-4a73-adce-a2a57f5bd87b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:35.675528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 204/404] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:35 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:35.682334 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01167398-f459-4a73-adce-a2a57f5bd87b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:35.726943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815585a-2cb7-4fc1-9407-9fa79a780c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:35.728829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815585a-2cb7-4fc1-9407-9fa79a780c06 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:35.729354 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815585a-2cb7-4fc1-9407-9fa79a780c06 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:35.729852 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1815585a-2cb7-4fc1-9407-9fa79a780c06 None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:35.731848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815585a-2cb7-4fc1-9407-9fa79a780c06 None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:35.733691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1815585a-2cb7-4fc1-9407-9fa79a780c06 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:35.737602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 201/405] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:35 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:35.744960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815585a-2cb7-4fc1-9407-9fa79a780c06 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:35.827104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:35.893770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01167398-f459-4a73-adce-a2a57f5bd87b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:35.906012 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1815585a-2cb7-4fc1-9407-9fa79a780c06 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:35.940589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce8033-b1b8-4758-a2fe-06b9a746db0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:35.941850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce8033-b1b8-4758-a2fe-06b9a746db0c None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:35.942347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce8033-b1b8-4758-a2fe-06b9a746db0c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:35.942791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5204bbb-3cd0-41a0-aa74-63ca13e8c9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:35.942908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce8033-b1b8-4758-a2fe-06b9a746db0c None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/groups/24f62c77801c46a5a9271f50cbac1161/roles/5f24daf2ba2b4f8fab95c87cefb52231` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:35.943864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204bbb-3cd0-41a0-aa74-63ca13e8c9f5 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:35.944278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204bbb-3cd0-41a0-aa74-63ca13e8c9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:35.944679 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5204bbb-3cd0-41a0-aa74-63ca13e8c9f5 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:35.945825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce8033-b1b8-4758-a2fe-06b9a746db0c None admin] RBAC: Authorizing `identity:create_grant(project_id=4088a75edb3f4c128c5f08a69f835d4f, group_id=24f62c77801c46a5a9271f50cbac1161, role_id=5f24daf2ba2b4f8fab95c87cefb52231)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:35.946103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5204bbb-3cd0-41a0-aa74-63ca13e8c9f5 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:35.947664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5204bbb-3cd0-41a0-aa74-63ca13e8c9f5 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:35.947776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce8033-b1b8-4758-a2fe-06b9a746db0c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:35.949783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 202/406] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:35 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:35.951066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91a7a6-00dd-4f17-91c7-ce9826d0b636 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:35.951477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91a7a6-00dd-4f17-91c7-ce9826d0b636 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:35.951821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91a7a6-00dd-4f17-91c7-ce9826d0b636 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:35.955417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 203/407] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:35.956874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91a7a6-00dd-4f17-91c7-ce9826d0b636 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:36.061637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 205/408] 10.222.0.81 () {58 vars in 1442 bytes} [Thu Jun 16 18:37:35 2022] PUT /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/groups/24f62c77801c46a5a9271f50cbac1161/roles/5f24daf2ba2b4f8fab95c87cefb52231 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:36.063642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672a3af-0a8e-4d8e-807d-624cf67823aa None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:36.064490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672a3af-0a8e-4d8e-807d-624cf67823aa None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:36.065009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672a3af-0a8e-4d8e-807d-624cf67823aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:36.068896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91a7a6-00dd-4f17-91c7-ce9826d0b636 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:36.082750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:36.098881 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b672a3af-0a8e-4d8e-807d-624cf67823aa None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:36.100430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b672a3af-0a8e-4d8e-807d-624cf67823aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:36.110644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147bce6-aba2-4c2f-bdb8-c1f052b0f55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:36.111537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147bce6-aba2-4c2f-bdb8-c1f052b0f55b None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:36.111825 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147bce6-aba2-4c2f-bdb8-c1f052b0f55b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:36.112145 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147bce6-aba2-4c2f-bdb8-c1f052b0f55b None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/groups/24f62c77801c46a5a9271f50cbac1161/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:36.114503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147bce6-aba2-4c2f-bdb8-c1f052b0f55b None admin] RBAC: Authorizing `identity:create_grant(project_id=4088a75edb3f4c128c5f08a69f835d4f, group_id=24f62c77801c46a5a9271f50cbac1161, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:36.119291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147bce6-aba2-4c2f-bdb8-c1f052b0f55b None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:36.149481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:36.194312 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 206/409] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:36.218664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672a3af-0a8e-4d8e-807d-624cf67823aa None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:36.226952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 204/410] 10.222.0.81 () {58 vars in 1442 bytes} [Thu Jun 16 18:37:35 2022] PUT /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f/groups/24f62c77801c46a5a9271f50cbac1161/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:36.284281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3f050-9553-4bc7-b84c-ab4d002cdd17 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:36.284623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3f050-9553-4bc7-b84c-ab4d002cdd17 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:36.286047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3f050-9553-4bc7-b84c-ab4d002cdd17 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:36.287644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 205/411] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:36.295018 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c00a9e-7a4b-4f15-afef-9c1b1316d619 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:36.295641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c00a9e-7a4b-4f15-afef-9c1b1316d619 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:36.296368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c00a9e-7a4b-4f15-afef-9c1b1316d619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:36.327291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-b4c00a9e-7a4b-4f15-afef-9c1b1316d619 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:36.328598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b4c00a9e-7a4b-4f15-afef-9c1b1316d619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:36.418680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672a3af-0a8e-4d8e-807d-624cf67823aa None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:36.461623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 206/412] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:36.486837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00baf84-a48e-4605-9c00-930dc158b94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:36.487651 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00baf84-a48e-4605-9c00-930dc158b94e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:36.487931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00baf84-a48e-4605-9c00-930dc158b94e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:36.488207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00baf84-a48e-4605-9c00-930dc158b94e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:36.496684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c00a9e-7a4b-4f15-afef-9c1b1316d619 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:36.499913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00baf84-a48e-4605-9c00-930dc158b94e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-f00baf84-a48e-4605-9c00-930dc158b94e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:36.509814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:36.510748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:36.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 207/413] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:37:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:36.518157 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00baf84-a48e-4605-9c00-930dc158b94e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:36.550815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:36.643936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00baf84-a48e-4605-9c00-930dc158b94e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:36.667568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:36.675332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c00a9e-7a4b-4f15-afef-9c1b1316d619 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:36.683363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca67df-a67e-40c9-b581-1bb11ca26d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:36.684044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca67df-a67e-40c9-b581-1bb11ca26d25 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:36.684330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca67df-a67e-40c9-b581-1bb11ca26d25 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:36.684603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca67df-a67e-40c9-b581-1bb11ca26d25 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:36.685381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca67df-a67e-40c9-b581-1bb11ca26d25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:36.685689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca67df-a67e-40c9-b581-1bb11ca26d25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:36.687079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca67df-a67e-40c9-b581-1bb11ca26d25 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:36.697975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 208/414] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:36.703921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca67df-a67e-40c9-b581-1bb11ca26d25 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:36.789961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b435dc5-9513-4590-95c7-5a1cbccfe138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:36.791285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b435dc5-9513-4590-95c7-5a1cbccfe138 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:36.791776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b435dc5-9513-4590-95c7-5a1cbccfe138 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:36.792278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b435dc5-9513-4590-95c7-5a1cbccfe138 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:36.797723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:36.818285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca67df-a67e-40c9-b581-1bb11ca26d25 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:36.835000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b435dc5-9513-4590-95c7-5a1cbccfe138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-1b435dc5-9513-4590-95c7-5a1cbccfe138 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:36.849740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:36.850750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:36.851560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 207/415] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:36.860877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b435dc5-9513-4590-95c7-5a1cbccfe138 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:36.910045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b8ce7-1863-4a5c-b112-71a0f2ca512d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:36.911240 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b8ce7-1863-4a5c-b112-71a0f2ca512d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:36.911713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b8ce7-1863-4a5c-b112-71a0f2ca512d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:36.912162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b8ce7-1863-4a5c-b112-71a0f2ca512d None admin] PATH_INFO: `/v3/groups/2dd9d6313e5a49e9a89359abfa2ba378` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:36.913665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b8ce7-1863-4a5c-b112-71a0f2ca512d None admin] RBAC: Authorizing `identity:get_group(group_id=2dd9d6313e5a49e9a89359abfa2ba378)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:36.915315 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b8ce7-1863-4a5c-b112-71a0f2ca512d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:36.917445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 209/416] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:36 2022] GET /identity/v3/groups/2dd9d6313e5a49e9a89359abfa2ba378 => generated 229 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:36.924423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b8ce7-1863-4a5c-b112-71a0f2ca512d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:37.017337 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b435dc5-9513-4590-95c7-5a1cbccfe138 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:37.085038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b8ce7-1863-4a5c-b112-71a0f2ca512d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:37.114237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3727f-ef73-477d-8922-56f700afe64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:37.114917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3727f-ef73-477d-8922-56f700afe64d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:37.115208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3727f-ef73-477d-8922-56f700afe64d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:37.115752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3727f-ef73-477d-8922-56f700afe64d None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:37.116504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3727f-ef73-477d-8922-56f700afe64d None admin] RBAC: Adding query filter params (name=member) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:37.116818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3727f-ef73-477d-8922-56f700afe64d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:37.118380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3727f-ef73-477d-8922-56f700afe64d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:37.123639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e8390-e955-46ed-a874-d0fd5bef65c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:37.124717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8390-e955-46ed-a874-d0fd5bef65c2 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:37.125219 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8390-e955-46ed-a874-d0fd5bef65c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:37.125686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8390-e955-46ed-a874-d0fd5bef65c2 None admin] PATH_INFO: `/v3/projects/3b167122bd2b4c3cba143eecf3576355` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:37.127428 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e8390-e955-46ed-a874-d0fd5bef65c2 None admin] RBAC: Authorizing `identity:get_project(project_id=3b167122bd2b4c3cba143eecf3576355)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:37.129590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e8390-e955-46ed-a874-d0fd5bef65c2 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:37.132994 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 210/417] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:36 2022] GET /identity/v3/projects/3b167122bd2b4c3cba143eecf3576355 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:37.136726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 208/418] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:37.141951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e8390-e955-46ed-a874-d0fd5bef65c2 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:37.142065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3727f-ef73-477d-8922-56f700afe64d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:37.167093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:37.198807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:37.255748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e8390-e955-46ed-a874-d0fd5bef65c2 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:37.299927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e7472-3a90-477a-8024-47e3e39a11cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:37.302014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7472-3a90-477a-8024-47e3e39a11cc None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:37.303319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7472-3a90-477a-8024-47e3e39a11cc None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:37.304898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7472-3a90-477a-8024-47e3e39a11cc None admin] PATH_INFO: `/v3/projects/3b167122bd2b4c3cba143eecf3576355/groups/2dd9d6313e5a49e9a89359abfa2ba378/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:37.307804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e7472-3a90-477a-8024-47e3e39a11cc None admin] RBAC: Authorizing `identity:create_grant(project_id=3b167122bd2b4c3cba143eecf3576355, group_id=2dd9d6313e5a49e9a89359abfa2ba378, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:37.309641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e7472-3a90-477a-8024-47e3e39a11cc None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:37.361591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3727f-ef73-477d-8922-56f700afe64d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:37.388573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 211/419] 10.222.0.81 () {58 vars in 1442 bytes} [Thu Jun 16 18:37:37 2022] PUT /identity/v3/projects/3b167122bd2b4c3cba143eecf3576355/groups/2dd9d6313e5a49e9a89359abfa2ba378/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:37.408793 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b90aaa-a57c-44c2-ad0b-5e42d0e2d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:37.409513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b90aaa-a57c-44c2-ad0b-5e42d0e2d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:37.409845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b90aaa-a57c-44c2-ad0b-5e42d0e2d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:37.410127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b90aaa-a57c-44c2-ad0b-5e42d0e2d9d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:37.489995 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b90aaa-a57c-44c2-ad0b-5e42d0e2d9d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:37.491679 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b90aaa-a57c-44c2-ad0b-5e42d0e2d9d0 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:37.520730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:37.603735 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 209/420] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:37:37.604380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bf51a-527f-4604-8e1d-a8b4cc1d7998 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:37.604673 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bf51a-527f-4604-8e1d-a8b4cc1d7998 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:37.605046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bf51a-527f-4604-8e1d-a8b4cc1d7998 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:37.607673 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 212/421] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:37.611590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b90aaa-a57c-44c2-ad0b-5e42d0e2d9d0 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:37.613363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118deed-c455-486c-8b9b-8e2bd21b83d5 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:37.613363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118deed-c455-486c-8b9b-8e2bd21b83d5 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:37.613530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118deed-c455-486c-8b9b-8e2bd21b83d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:37.633183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-4118deed-c455-486c-8b9b-8e2bd21b83d5 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:37.635346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-4118deed-c455-486c-8b9b-8e2bd21b83d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:37.711924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b90aaa-a57c-44c2-ad0b-5e42d0e2d9d0 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:37.715065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 213/422] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:37.737337 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:37.739149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118deed-c455-486c-8b9b-8e2bd21b83d5 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:37.750070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66927124-f768-4179-8b93-2d402dae95ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:37.750862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66927124-f768-4179-8b93-2d402dae95ef None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:37.751137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66927124-f768-4179-8b93-2d402dae95ef None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:37.751410 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66927124-f768-4179-8b93-2d402dae95ef None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:37.752602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66927124-f768-4179-8b93-2d402dae95ef None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:37.754061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66927124-f768-4179-8b93-2d402dae95ef None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:37.760105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 210/423] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:37 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:37.761116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66927124-f768-4179-8b93-2d402dae95ef None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:37.762124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:37.907186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118deed-c455-486c-8b9b-8e2bd21b83d5 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:37.959911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf97fd6-83ac-49fc-8cf7-26cd76023fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:37.961468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf97fd6-83ac-49fc-8cf7-26cd76023fc3 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:37.961757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf97fd6-83ac-49fc-8cf7-26cd76023fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:37.962083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf97fd6-83ac-49fc-8cf7-26cd76023fc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:37.966758 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66927124-f768-4179-8b93-2d402dae95ef None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:37.984380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf97fd6-83ac-49fc-8cf7-26cd76023fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-0cf97fd6-83ac-49fc-8cf7-26cd76023fc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:37.994251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:37.995128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:37.995924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 214/424] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:37:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:38.010057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf97fd6-83ac-49fc-8cf7-26cd76023fc3 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:38.039074 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81182ce-93b9-48e9-aa5f-fa7128052ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:38.039940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81182ce-93b9-48e9-aa5f-fa7128052ebd None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:38.040988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81182ce-93b9-48e9-aa5f-fa7128052ebd None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:38.041344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81182ce-93b9-48e9-aa5f-fa7128052ebd None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:38.043834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81182ce-93b9-48e9-aa5f-fa7128052ebd None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:38.047678 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81182ce-93b9-48e9-aa5f-fa7128052ebd None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:38.052902 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 211/425] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:37 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:38.058638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81182ce-93b9-48e9-aa5f-fa7128052ebd None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:38.129986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:38.178635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81182ce-93b9-48e9-aa5f-fa7128052ebd None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:38.179905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:38.219089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8d6f0-23a7-440c-87d3-e82199856c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:38.219749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8d6f0-23a7-440c-87d3-e82199856c88 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:38.220094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8d6f0-23a7-440c-87d3-e82199856c88 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:38.220369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8d6f0-23a7-440c-87d3-e82199856c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:38.221268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8d6f0-23a7-440c-87d3-e82199856c88 None admin] RBAC: Adding query filter params (group.id=24f62c77801c46a5a9271f50cbac1161, role.id=0960434fd9da404ea96841eb4306eaae, scope.project.id=0e628770474b423e99809dd568e34a0e) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:38.221579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8d6f0-23a7-440c-87d3-e82199856c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:38.223445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8d6f0-23a7-440c-87d3-e82199856c88 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:38.230832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 212/426] 10.222.0.81 () {56 vars in 1376 bytes} [Thu Jun 16 18:37:38 2022] GET /identity/v3/role_assignments?group.id=24f62c77801c46a5a9271f50cbac1161&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=0960434fd9da404ea96841eb4306eaae => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:38.246164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf97fd6-83ac-49fc-8cf7-26cd76023fc3 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:38.282825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6733d-58d2-49cf-b16a-92d48d197490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:38.283423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6733d-58d2-49cf-b16a-92d48d197490 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:38.283705 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6733d-58d2-49cf-b16a-92d48d197490 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:38.283986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6733d-58d2-49cf-b16a-92d48d197490 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:38.284746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6733d-58d2-49cf-b16a-92d48d197490 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:38.285053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6733d-58d2-49cf-b16a-92d48d197490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:38.286524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6733d-58d2-49cf-b16a-92d48d197490 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:38.296562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 215/427] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:38.302073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8d6f0-23a7-440c-87d3-e82199856c88 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:38.464565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8d6f0-23a7-440c-87d3-e82199856c88 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:38.480336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafc297-4c0c-4c56-9214-d1ecce8a361f None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:38.480765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafc297-4c0c-4c56-9214-d1ecce8a361f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:38.481155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafc297-4c0c-4c56-9214-d1ecce8a361f None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:38.482639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 216/428] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:38.488781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c90a5-1466-43d2-b6f3-17a4e28081c8 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:38.489070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c90a5-1466-43d2-b6f3-17a4e28081c8 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:38.490129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c90a5-1466-43d2-b6f3-17a4e28081c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:38.490245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:38.502488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3db018-bcb2-497d-adfa-c24f8b803da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:38.503453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db018-bcb2-497d-adfa-c24f8b803da0 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:38.503952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db018-bcb2-497d-adfa-c24f8b803da0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:38.504436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3db018-bcb2-497d-adfa-c24f8b803da0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:38.518063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-a35c90a5-1466-43d2-b6f3-17a4e28081c8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:38.519817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a35c90a5-1466-43d2-b6f3-17a4e28081c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:38.586510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3db018-bcb2-497d-adfa-c24f8b803da0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:38.588954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3db018-bcb2-497d-adfa-c24f8b803da0 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:38.608729 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 217/429] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:38.632610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c90a5-1466-43d2-b6f3-17a4e28081c8 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:38.702926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 213/430] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:38.708764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3db018-bcb2-497d-adfa-c24f8b803da0 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:38.711776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:38.734501 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:38.760212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c90a5-1466-43d2-b6f3-17a4e28081c8 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:38.800253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c584bed-af5b-4c23-8c5b-1fe3e7334565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:38.801075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c584bed-af5b-4c23-8c5b-1fe3e7334565 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:38.801400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c584bed-af5b-4c23-8c5b-1fe3e7334565 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:38.801675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c584bed-af5b-4c23-8c5b-1fe3e7334565 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:38.812385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c584bed-af5b-4c23-8c5b-1fe3e7334565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:38.819528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3db018-bcb2-497d-adfa-c24f8b803da0 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-1c584bed-af5b-4c23-8c5b-1fe3e7334565 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:38.820432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:38.821382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:38.822719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 218/431] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:37:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:38.827927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c584bed-af5b-4c23-8c5b-1fe3e7334565 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:38.862073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b51d-0b8f-4faf-8a4a-4343e5f56c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:38.863330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b51d-0b8f-4faf-8a4a-4343e5f56c4d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:38.863816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b51d-0b8f-4faf-8a4a-4343e5f56c4d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:38.864328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b51d-0b8f-4faf-8a4a-4343e5f56c4d None admin] PATH_INFO: `/v3/projects/0e628770474b423e99809dd568e34a0e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:38.866030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8b51d-0b8f-4faf-8a4a-4343e5f56c4d None admin] RBAC: Authorizing `identity:get_project(project_id=0e628770474b423e99809dd568e34a0e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:38.867685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8b51d-0b8f-4faf-8a4a-4343e5f56c4d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:38.870665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 214/432] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:38 2022] GET /identity/v3/projects/0e628770474b423e99809dd568e34a0e => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:38.876835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b51d-0b8f-4faf-8a4a-4343e5f56c4d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:38.938811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c584bed-af5b-4c23-8c5b-1fe3e7334565 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:38.966916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:38.980601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d66c57-3b94-453e-9a1f-b0870245f047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:38.981552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66c57-3b94-453e-9a1f-b0870245f047 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:38.982098 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66c57-3b94-453e-9a1f-b0870245f047 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:38.982423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d66c57-3b94-453e-9a1f-b0870245f047 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:38.983200 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d66c57-3b94-453e-9a1f-b0870245f047 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:38.983512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d66c57-3b94-453e-9a1f-b0870245f047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:38.984625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:38.984944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d66c57-3b94-453e-9a1f-b0870245f047 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:38.995660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 219/433] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:38.997468 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b51d-0b8f-4faf-8a4a-4343e5f56c4d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:39.002105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d66c57-3b94-453e-9a1f-b0870245f047 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:39.040729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349db8a7-d168-4119-b50a-c66047fce51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:39.042076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349db8a7-d168-4119-b50a-c66047fce51a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.042567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349db8a7-d168-4119-b50a-c66047fce51a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.043063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349db8a7-d168-4119-b50a-c66047fce51a None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.044476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349db8a7-d168-4119-b50a-c66047fce51a None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:39.046264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349db8a7-d168-4119-b50a-c66047fce51a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:39.048407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 215/434] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:38 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:39.054611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349db8a7-d168-4119-b50a-c66047fce51a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:39.120288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d66c57-3b94-453e-9a1f-b0870245f047 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:39.168189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3f7ef-d1ba-4c10-8d3d-db9d120d43ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:39.168995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3f7ef-d1ba-4c10-8d3d-db9d120d43ce None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.169503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3f7ef-d1ba-4c10-8d3d-db9d120d43ce None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.169604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3f7ef-d1ba-4c10-8d3d-db9d120d43ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.183361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349db8a7-d168-4119-b50a-c66047fce51a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:39.217668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78320340-ce8b-41d0-bc9c-a487359d91bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:39.218313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78320340-ce8b-41d0-bc9c-a487359d91bf None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.218596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78320340-ce8b-41d0-bc9c-a487359d91bf None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.218864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78320340-ce8b-41d0-bc9c-a487359d91bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.219678 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78320340-ce8b-41d0-bc9c-a487359d91bf None admin] RBAC: Adding query filter params (group.id=24f62c77801c46a5a9271f50cbac1161, role.id=5f24daf2ba2b4f8fab95c87cefb52231, scope.project.id=0e628770474b423e99809dd568e34a0e) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:39.219990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78320340-ce8b-41d0-bc9c-a487359d91bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:39.221449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78320340-ce8b-41d0-bc9c-a487359d91bf None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:39.225639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:39.233898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 216/435] 10.222.0.81 () {56 vars in 1376 bytes} [Thu Jun 16 18:37:39 2022] GET /identity/v3/role_assignments?group.id=24f62c77801c46a5a9271f50cbac1161&scope.project.id=0e628770474b423e99809dd568e34a0e&role.id=5f24daf2ba2b4f8fab95c87cefb52231 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:39.233898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccf2c-689c-4af9-b63e-0f3804fc86f5 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.233898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccf2c-689c-4af9-b63e-0f3804fc86f5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.233898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ccf2c-689c-4af9-b63e-0f3804fc86f5 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.233898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 217/436] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:39.235558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050ca66-22e1-4183-b55c-34f3d024df90 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.235848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050ca66-22e1-4183-b55c-34f3d024df90 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.236123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050ca66-22e1-4183-b55c-34f3d024df90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.254590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3f7ef-d1ba-4c10-8d3d-db9d120d43ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:39.256433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3f7ef-d1ba-4c10-8d3d-db9d120d43ce None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:39.257092 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4050ca66-22e1-4183-b55c-34f3d024df90 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:39.258379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4050ca66-22e1-4183-b55c-34f3d024df90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:39.286621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:39.373400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 218/437] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:39.387136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 220/438] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:39.392554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050ca66-22e1-4183-b55c-34f3d024df90 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:39.402321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3f7ef-d1ba-4c10-8d3d-db9d120d43ce None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:39.504868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:39.515261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050ca66-22e1-4183-b55c-34f3d024df90 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:39.519782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3f7ef-d1ba-4c10-8d3d-db9d120d43ce None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:39.561918 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13462b22-5fff-4aea-9b10-e2f62ec8ece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:39.563457 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13462b22-5fff-4aea-9b10-e2f62ec8ece0 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.564205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13462b22-5fff-4aea-9b10-e2f62ec8ece0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.565118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13462b22-5fff-4aea-9b10-e2f62ec8ece0 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.565915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e33f0-205c-437b-8b41-e67edf3b734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:39.566713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e33f0-205c-437b-8b41-e67edf3b734b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.566995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e33f0-205c-437b-8b41-e67edf3b734b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.567265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e33f0-205c-437b-8b41-e67edf3b734b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.569066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13462b22-5fff-4aea-9b10-e2f62ec8ece0 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:39.570538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13462b22-5fff-4aea-9b10-e2f62ec8ece0 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:39.572584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 219/439] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:39 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:39.578188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13462b22-5fff-4aea-9b10-e2f62ec8ece0 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:39.583989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e33f0-205c-437b-8b41-e67edf3b734b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-de7e33f0-205c-437b-8b41-e67edf3b734b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:39.592104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:39.593002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:39.593805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 221/440] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:39.601374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e33f0-205c-437b-8b41-e67edf3b734b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:39.606504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:39.708300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13462b22-5fff-4aea-9b10-e2f62ec8ece0 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:39.724212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e33f0-205c-437b-8b41-e67edf3b734b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:39.747890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1c2c3-bd32-4855-ab48-bd87dd0b1065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:39.748744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c2c3-bd32-4855-ab48-bd87dd0b1065 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.749028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c2c3-bd32-4855-ab48-bd87dd0b1065 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.749333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1c2c3-bd32-4855-ab48-bd87dd0b1065 None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.750499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1c2c3-bd32-4855-ab48-bd87dd0b1065 None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:39.751933 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1c2c3-bd32-4855-ab48-bd87dd0b1065 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:39.758239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 220/441] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:39 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:39.759398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1c2c3-bd32-4855-ab48-bd87dd0b1065 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:39.764626 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f208571-94b3-4a7f-a648-47415d625a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:39.765320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f208571-94b3-4a7f-a648-47415d625a6b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.765608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f208571-94b3-4a7f-a648-47415d625a6b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.765884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f208571-94b3-4a7f-a648-47415d625a6b None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.766624 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f208571-94b3-4a7f-a648-47415d625a6b None admin] RBAC: Adding query filter params (name=member) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:39.766933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f208571-94b3-4a7f-a648-47415d625a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:39.768297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f208571-94b3-4a7f-a648-47415d625a6b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:39.782133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 222/442] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:37:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:39.783384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f208571-94b3-4a7f-a648-47415d625a6b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:39.793988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:39.855481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:39.868841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1c2c3-bd32-4855-ab48-bd87dd0b1065 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:39.900454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f208571-94b3-4a7f-a648-47415d625a6b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:39.908051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3b62b-b585-4268-943c-f37d07027cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:39.908702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3b62b-b585-4268-943c-f37d07027cfd None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.908982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3b62b-b585-4268-943c-f37d07027cfd None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.909268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3b62b-b585-4268-943c-f37d07027cfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.910259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3b62b-b585-4268-943c-f37d07027cfd None admin] RBAC: Adding query filter params (group.id=24f62c77801c46a5a9271f50cbac1161, role.id=5f24daf2ba2b4f8fab95c87cefb52231, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:39.910446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3b62b-b585-4268-943c-f37d07027cfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:39.912040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3b62b-b585-4268-943c-f37d07027cfd None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:39.925459 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 221/443] 10.222.0.81 () {56 vars in 1376 bytes} [Thu Jun 16 18:37:39 2022] GET /identity/v3/role_assignments?group.id=24f62c77801c46a5a9271f50cbac1161&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=5f24daf2ba2b4f8fab95c87cefb52231 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:39.927020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e76e-3be8-47c5-840b-1c94be364cec None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.927318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e76e-3be8-47c5-840b-1c94be364cec None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.927603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e76e-3be8-47c5-840b-1c94be364cec None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.929339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 222/444] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:39.935106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548ee58-3950-4db8-8df3-b2f02875482d None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.935411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548ee58-3950-4db8-8df3-b2f02875482d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.935662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548ee58-3950-4db8-8df3-b2f02875482d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.939541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ca402-1068-4b79-b265-a2c1b838bc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:39.940222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ca402-1068-4b79-b265-a2c1b838bc39 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:39.940516 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ca402-1068-4b79-b265-a2c1b838bc39 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:39.940797 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ca402-1068-4b79-b265-a2c1b838bc39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:39.953526 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-8548ee58-3950-4db8-8df3-b2f02875482d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:39.954786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8548ee58-3950-4db8-8df3-b2f02875482d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:40.022652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ca402-1068-4b79-b265-a2c1b838bc39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:40.024058 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ca402-1068-4b79-b265-a2c1b838bc39 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:40.033284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 223/445] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:40.034388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:40.053765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548ee58-3950-4db8-8df3-b2f02875482d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:40.135153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 223/446] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:37:40.140447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ca402-1068-4b79-b265-a2c1b838bc39 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:40.149062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:40.166946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548ee58-3950-4db8-8df3-b2f02875482d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:40.199829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58899045-e7f4-4c2e-839c-de0e0df17a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:40.200628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58899045-e7f4-4c2e-839c-de0e0df17a59 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:40.200914 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58899045-e7f4-4c2e-839c-de0e0df17a59 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:40.201191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58899045-e7f4-4c2e-839c-de0e0df17a59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:40.211052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58899045-e7f4-4c2e-839c-de0e0df17a59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-58899045-e7f4-4c2e-839c-de0e0df17a59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:40.219306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:40.220180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:40.221014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 224/447] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:37:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:40.226126 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58899045-e7f4-4c2e-839c-de0e0df17a59 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:40.235077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ca402-1068-4b79-b265-a2c1b838bc39 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:40.242457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:40.269374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577a3a4-3681-43ad-991f-2c273e89ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:40.270354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577a3a4-3681-43ad-991f-2c273e89ae14 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:40.270557 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577a3a4-3681-43ad-991f-2c273e89ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:40.270824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9577a3a4-3681-43ad-991f-2c273e89ae14 None admin] PATH_INFO: `/v3/projects/4088a75edb3f4c128c5f08a69f835d4f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:40.271995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577a3a4-3681-43ad-991f-2c273e89ae14 None admin] RBAC: Authorizing `identity:get_project(project_id=4088a75edb3f4c128c5f08a69f835d4f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:40.273520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9577a3a4-3681-43ad-991f-2c273e89ae14 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:40.275788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 224/448] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:40 2022] GET /identity/v3/projects/4088a75edb3f4c128c5f08a69f835d4f => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:40.281054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577a3a4-3681-43ad-991f-2c273e89ae14 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:40.324611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58899045-e7f4-4c2e-839c-de0e0df17a59 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:40.358336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d7500-9916-47b9-8884-c77e434f4137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:40.359024 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d7500-9916-47b9-8884-c77e434f4137 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:40.359312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d7500-9916-47b9-8884-c77e434f4137 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:40.359596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d7500-9916-47b9-8884-c77e434f4137 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:40.360338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d7500-9916-47b9-8884-c77e434f4137 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:40.360651 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d7500-9916-47b9-8884-c77e434f4137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:40.362154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d7500-9916-47b9-8884-c77e434f4137 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:40.370825 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 225/449] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:40.376312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d7500-9916-47b9-8884-c77e434f4137 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:40.379105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9577a3a4-3681-43ad-991f-2c273e89ae14 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:40.379630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:40.413063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd7c98-984c-4218-adef-013abff5bbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:40.413931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd7c98-984c-4218-adef-013abff5bbfc None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:40.414209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd7c98-984c-4218-adef-013abff5bbfc None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:40.414475 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd7c98-984c-4218-adef-013abff5bbfc None admin] PATH_INFO: `/v3/groups/24f62c77801c46a5a9271f50cbac1161` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:40.415635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd7c98-984c-4218-adef-013abff5bbfc None admin] RBAC: Authorizing `identity:get_group(group_id=24f62c77801c46a5a9271f50cbac1161)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:40.417055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd7c98-984c-4218-adef-013abff5bbfc None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:40.418962 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 225/450] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:40 2022] GET /identity/v3/groups/24f62c77801c46a5a9271f50cbac1161 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:40.424414 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd7c98-984c-4218-adef-013abff5bbfc None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:40.466355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:40.478464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d7500-9916-47b9-8884-c77e434f4137 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:40.511969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60419eaf-b69b-4588-b440-45e6cb9acbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:40.512610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419eaf-b69b-4588-b440-45e6cb9acbd9 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:40.512891 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419eaf-b69b-4588-b440-45e6cb9acbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:40.513162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419eaf-b69b-4588-b440-45e6cb9acbd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:40.520114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd7c98-984c-4218-adef-013abff5bbfc None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:40.553980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb09477-0ced-419e-b87c-3e0d41837fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:40.554565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb09477-0ced-419e-b87c-3e0d41837fcf None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:40.554872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb09477-0ced-419e-b87c-3e0d41837fcf None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:40.555153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb09477-0ced-419e-b87c-3e0d41837fcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:40.555974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb09477-0ced-419e-b87c-3e0d41837fcf None admin] RBAC: Adding query filter params (group.id=24f62c77801c46a5a9271f50cbac1161, role.id=0960434fd9da404ea96841eb4306eaae, scope.project.id=4088a75edb3f4c128c5f08a69f835d4f) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:40.556289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb09477-0ced-419e-b87c-3e0d41837fcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:40.557766 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb09477-0ced-419e-b87c-3e0d41837fcf None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:40.563401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 226/451] 10.222.0.81 () {56 vars in 1376 bytes} [Thu Jun 16 18:37:40 2022] GET /identity/v3/role_assignments?group.id=24f62c77801c46a5a9271f50cbac1161&scope.project.id=4088a75edb3f4c128c5f08a69f835d4f&role.id=0960434fd9da404ea96841eb4306eaae => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:40.577994 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60419eaf-b69b-4588-b440-45e6cb9acbd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:40.579487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60419eaf-b69b-4588-b440-45e6cb9acbd9 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:40.582712 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:40.683259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 226/452] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 18:37:40.688976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb09477-0ced-419e-b87c-3e0d41837fcf None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:40.791602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb09477-0ced-419e-b87c-3e0d41837fcf None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:40.798519 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:40.825026 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52e91a-f2c0-4d24-a78e-a677240a21e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:40.825920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52e91a-f2c0-4d24-a78e-a677240a21e9 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:40.826208 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52e91a-f2c0-4d24-a78e-a677240a21e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:40.826484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52e91a-f2c0-4d24-a78e-a677240a21e9 None admin] PATH_INFO: `/v3/projects/3b167122bd2b4c3cba143eecf3576355` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:40.827880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52e91a-f2c0-4d24-a78e-a677240a21e9 None admin] RBAC: Authorizing `identity:get_project(project_id=3b167122bd2b4c3cba143eecf3576355)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:40.829846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52e91a-f2c0-4d24-a78e-a677240a21e9 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:40.831538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 227/453] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:37:40 2022] GET /identity/v3/projects/3b167122bd2b4c3cba143eecf3576355 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:40.837048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60419eaf-b69b-4588-b440-45e6cb9acbd9 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:40.935922 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60419eaf-b69b-4588-b440-45e6cb9acbd9 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:40.938259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:40.970064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755fc76-1d2b-41bf-93c1-615cd961b028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:40.970910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755fc76-1d2b-41bf-93c1-615cd961b028 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:40.971186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755fc76-1d2b-41bf-93c1-615cd961b028 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:40.971458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755fc76-1d2b-41bf-93c1-615cd961b028 None admin] PATH_INFO: `/v3/groups/2dd9d6313e5a49e9a89359abfa2ba378` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:40.972616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755fc76-1d2b-41bf-93c1-615cd961b028 None admin] RBAC: Authorizing `identity:get_group(group_id=2dd9d6313e5a49e9a89359abfa2ba378)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:40.974058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755fc76-1d2b-41bf-93c1-615cd961b028 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:40.975648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 227/454] 10.222.0.81 () {56 vars in 1180 bytes} [Thu Jun 16 18:37:40 2022] GET /identity/v3/groups/2dd9d6313e5a49e9a89359abfa2ba378 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:40.981818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52e91a-f2c0-4d24-a78e-a677240a21e9 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:41.084173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52e91a-f2c0-4d24-a78e-a677240a21e9 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:41.117036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f304a-df2d-4fe9-9e48-14c24e716fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:41.117714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f304a-df2d-4fe9-9e48-14c24e716fff None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:41.118001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f304a-df2d-4fe9-9e48-14c24e716fff None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:41.118278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f304a-df2d-4fe9-9e48-14c24e716fff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:41.119661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f304a-df2d-4fe9-9e48-14c24e716fff None admin] RBAC: Adding query filter params (group.id=2dd9d6313e5a49e9a89359abfa2ba378, role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=3b167122bd2b4c3cba143eecf3576355) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:41.119973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f304a-df2d-4fe9-9e48-14c24e716fff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:41.121430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f304a-df2d-4fe9-9e48-14c24e716fff None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:41.127135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 228/455] 10.222.0.81 () {56 vars in 1376 bytes} [Thu Jun 16 18:37:40 2022] GET /identity/v3/role_assignments?group.id=2dd9d6313e5a49e9a89359abfa2ba378&scope.project.id=3b167122bd2b4c3cba143eecf3576355&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:43.770610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b704363-a9f1-4cc9-91f2-e9ad27da7db3 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:43.771097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b704363-a9f1-4cc9-91f2-e9ad27da7db3 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:43.771097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b704363-a9f1-4cc9-91f2-e9ad27da7db3 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:43.772668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 228/456] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:43.778685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194599-1094-4b72-ba60-2f7c0eb23e51 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:43.779497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194599-1094-4b72-ba60-2f7c0eb23e51 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:43.779976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa194599-1094-4b72-ba60-2f7c0eb23e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:43.802203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-aa194599-1094-4b72-ba60-2f7c0eb23e51 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:43.803741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-aa194599-1094-4b72-ba60-2f7c0eb23e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:43.839385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:43.887961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 229/457] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:43.924819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b704363-a9f1-4cc9-91f2-e9ad27da7db3 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:44.001268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27190de-ec0e-4bbb-be2a-c571359ccb3d None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.001947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27190de-ec0e-4bbb-be2a-c571359ccb3d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.002429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27190de-ec0e-4bbb-be2a-c571359ccb3d None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.004354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 230/458] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:44.010554 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb57fc-5086-4e80-84de-9bdab3a1c62d None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.011137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb57fc-5086-4e80-84de-9bdab3a1c62d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.011608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb57fc-5086-4e80-84de-9bdab3a1c62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.041144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-84cb57fc-5086-4e80-84de-9bdab3a1c62d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:44.042360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-84cb57fc-5086-4e80-84de-9bdab3a1c62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:44.074979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b704363-a9f1-4cc9-91f2-e9ad27da7db3 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:44.117996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec237971-ce78-4beb-8815-dd14e2990e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:44.119331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec237971-ce78-4beb-8815-dd14e2990e5d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.119904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec237971-ce78-4beb-8815-dd14e2990e5d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.120189 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec237971-ce78-4beb-8815-dd14e2990e5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.132449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 231/459] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:44.133711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec237971-ce78-4beb-8815-dd14e2990e5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:44.135716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec237971-ce78-4beb-8815-dd14e2990e5d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:44.141986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22449854-25f2-4657-b4e8-fc357891840e None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.142284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22449854-25f2-4657-b4e8-fc357891840e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.142568 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22449854-25f2-4657-b4e8-fc357891840e None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.144341 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 232/460] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:44 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:44.149961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396d579-4870-4f7b-8db5-6f582a5b9f8b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.150818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396d579-4870-4f7b-8db5-6f582a5b9f8b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.151276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396d579-4870-4f7b-8db5-6f582a5b9f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-ec237971-ce78-4beb-8815-dd14e2990e5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:44.158593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:44.159443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:44.160259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:44.160259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:44.160259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:44.160259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:44.160259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 229/461] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:44.165471 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec237971-ce78-4beb-8815-dd14e2990e5d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:44.179523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2396d579-4870-4f7b-8db5-6f582a5b9f8b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:44.180850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2396d579-4870-4f7b-8db5-6f582a5b9f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:44.199518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:44.260633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 233/462] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:44.261779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396d579-4870-4f7b-8db5-6f582a5b9f8b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:44.304708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec237971-ce78-4beb-8815-dd14e2990e5d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:44.308864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:44.345371 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1baa5-238c-437a-918c-5e3cbcae0b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:44.346063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1baa5-238c-437a-918c-5e3cbcae0b78 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.346545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1baa5-238c-437a-918c-5e3cbcae0b78 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.346829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1baa5-238c-437a-918c-5e3cbcae0b78 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.347707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1baa5-238c-437a-918c-5e3cbcae0b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:44.348026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1baa5-238c-437a-918c-5e3cbcae0b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:44.349493 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1baa5-238c-437a-918c-5e3cbcae0b78 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:44.367777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 230/463] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:44.368765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1baa5-238c-437a-918c-5e3cbcae0b78 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:44.402370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396d579-4870-4f7b-8db5-6f582a5b9f8b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:44.437127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045acf7-39fb-4821-95bb-c34a1df72c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:44.437957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045acf7-39fb-4821-95bb-c34a1df72c94 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.438243 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045acf7-39fb-4821-95bb-c34a1df72c94 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.438517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045acf7-39fb-4821-95bb-c34a1df72c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.449133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045acf7-39fb-4821-95bb-c34a1df72c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:44.450678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045acf7-39fb-4821-95bb-c34a1df72c94 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:44.458586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-5045acf7-39fb-4821-95bb-c34a1df72c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:44.468852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:44.473869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:44.474693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:44.474693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:44.474693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:44.474693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:44.474693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 234/464] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:44.474693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045acf7-39fb-4821-95bb-c34a1df72c94 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:44.479783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1baa5-238c-437a-918c-5e3cbcae0b78 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:44.516368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732d2d2-c446-43cb-850e-bacf88418f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:44.517163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732d2d2-c446-43cb-850e-bacf88418f5e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.517487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732d2d2-c446-43cb-850e-bacf88418f5e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.517760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732d2d2-c446-43cb-850e-bacf88418f5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.528941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732d2d2-c446-43cb-850e-bacf88418f5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:44.530439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732d2d2-c446-43cb-850e-bacf88418f5e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-5732d2d2-c446-43cb-850e-bacf88418f5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:44.548416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:44.549307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:44.550173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:44.550173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:44.550173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:44.550173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:44.550173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 231/465] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:44.550876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1303c-c1a1-4fbc-a619-22ae60294ea4 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.551163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1303c-c1a1-4fbc-a619-22ae60294ea4 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.551382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1303c-c1a1-4fbc-a619-22ae60294ea4 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.556890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 232/466] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:44.556890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1303c-c1a1-4fbc-a619-22ae60294ea4 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:44.568951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:44.589666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045acf7-39fb-4821-95bb-c34a1df72c94 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:44.624462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09d11a-ceb1-46dc-9777-48b824d143b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:44.625131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09d11a-ceb1-46dc-9777-48b824d143b0 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.625451 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09d11a-ceb1-46dc-9777-48b824d143b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.625727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09d11a-ceb1-46dc-9777-48b824d143b0 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.626416 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09d11a-ceb1-46dc-9777-48b824d143b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:44.627835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09d11a-ceb1-46dc-9777-48b824d143b0 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:44.663270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1303c-c1a1-4fbc-a619-22ae60294ea4 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:44.695840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac0018-4fd8-4573-a33b-dbf64f75d8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:44.696506 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0018-4fd8-4573-a33b-dbf64f75d8d1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.696789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0018-4fd8-4573-a33b-dbf64f75d8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.697619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0018-4fd8-4573-a33b-dbf64f75d8d1 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.698341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac0018-4fd8-4573-a33b-dbf64f75d8d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:44.698653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac0018-4fd8-4573-a33b-dbf64f75d8d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:44.700040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac0018-4fd8-4573-a33b-dbf64f75d8d1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:44.702433 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:44.711560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 233/467] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:44.712352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac0018-4fd8-4573-a33b-dbf64f75d8d1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:44.731367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 235/468] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:37:44 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:44.732976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd431a-3e65-4fd6-95a1-ac41ba2e4a8b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.733455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd431a-3e65-4fd6-95a1-ac41ba2e4a8b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.734041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafd431a-3e65-4fd6-95a1-ac41ba2e4a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.752329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-bafd431a-3e65-4fd6-95a1-ac41ba2e4a8b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:44.754410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-bafd431a-3e65-4fd6-95a1-ac41ba2e4a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:44.843911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 236/469] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:44.845583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd431a-3e65-4fd6-95a1-ac41ba2e4a8b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:44.856204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac0018-4fd8-4573-a33b-dbf64f75d8d1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:44.904207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db99f85-b7c7-4cea-9a99-ebd751741463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:44.904942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db99f85-b7c7-4cea-9a99-ebd751741463 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:44.905228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db99f85-b7c7-4cea-9a99-ebd751741463 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:44.905595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db99f85-b7c7-4cea-9a99-ebd751741463 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:44.906327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db99f85-b7c7-4cea-9a99-ebd751741463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:44.906641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db99f85-b7c7-4cea-9a99-ebd751741463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:44.908028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db99f85-b7c7-4cea-9a99-ebd751741463 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:44.922643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 234/470] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:44.923472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db99f85-b7c7-4cea-9a99-ebd751741463 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:44.942951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:44.988596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:44.996103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafd431a-3e65-4fd6-95a1-ac41ba2e4a8b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:45.047781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d412f6c-cfec-4210-a6dc-35ae6862323d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:45.048806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d412f6c-cfec-4210-a6dc-35ae6862323d None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:45.049344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d412f6c-cfec-4210-a6dc-35ae6862323d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:45.049884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d412f6c-cfec-4210-a6dc-35ae6862323d None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:45.050790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d412f6c-cfec-4210-a6dc-35ae6862323d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:45.051465 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db99f85-b7c7-4cea-9a99-ebd751741463 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:45.052665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d412f6c-cfec-4210-a6dc-35ae6862323d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:45.089493 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b566f3-5fd3-4ddd-948b-35b110829de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:45.090278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b566f3-5fd3-4ddd-948b-35b110829de8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:45.090556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b566f3-5fd3-4ddd-948b-35b110829de8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:45.090827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b566f3-5fd3-4ddd-948b-35b110829de8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:45.105921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b566f3-5fd3-4ddd-948b-35b110829de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:45.105921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b566f3-5fd3-4ddd-948b-35b110829de8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-c5b566f3-5fd3-4ddd-948b-35b110829de8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:45.120965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:45.121861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:45.122726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:45.122726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:45.122726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:45.122726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:45.122726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 235/471] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:45.122999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b566f3-5fd3-4ddd-948b-35b110829de8 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:45.134053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 237/472] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:37:44 2022] POST /identity/v3/users => generated 246 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:45.134053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087d0f0-c949-4803-9fe6-229df3dc45fe None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:45.134053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087d0f0-c949-4803-9fe6-229df3dc45fe None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:45.134053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087d0f0-c949-4803-9fe6-229df3dc45fe None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:45.142091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 238/473] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:45.143046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087d0f0-c949-4803-9fe6-229df3dc45fe None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:45.245659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b566f3-5fd3-4ddd-948b-35b110829de8 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:45.255597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:45.265683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087d0f0-c949-4803-9fe6-229df3dc45fe None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:45.282294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950928b-c7c7-4337-bdbb-7f7eec7d43ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:45.282975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950928b-c7c7-4337-bdbb-7f7eec7d43ca None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:45.283257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950928b-c7c7-4337-bdbb-7f7eec7d43ca None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:45.283544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950928b-c7c7-4337-bdbb-7f7eec7d43ca None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:45.284250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950928b-c7c7-4337-bdbb-7f7eec7d43ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:45.285761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950928b-c7c7-4337-bdbb-7f7eec7d43ca None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:45.293936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:45.308680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2649cd8-82d2-40b7-a837-e98dd9b3d119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:45.309639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2649cd8-82d2-40b7-a837-e98dd9b3d119 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:45.310079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2649cd8-82d2-40b7-a837-e98dd9b3d119 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:45.310506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2649cd8-82d2-40b7-a837-e98dd9b3d119 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:45.311417 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2649cd8-82d2-40b7-a837-e98dd9b3d119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:45.311878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2649cd8-82d2-40b7-a837-e98dd9b3d119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:45.313584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2649cd8-82d2-40b7-a837-e98dd9b3d119 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:45.333877 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 239/474] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:45.338548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753290e-4d6a-4ad8-90a2-e0c2676599e3 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:45.338548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753290e-4d6a-4ad8-90a2-e0c2676599e3 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:45.338548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753290e-4d6a-4ad8-90a2-e0c2676599e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:45.353143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6753290e-4d6a-4ad8-90a2-e0c2676599e3 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:45.356164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6753290e-4d6a-4ad8-90a2-e0c2676599e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:45.396426 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 236/475] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:37:45 2022] POST /identity/v3/users => generated 248 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:45.397420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950928b-c7c7-4337-bdbb-7f7eec7d43ca None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:45.455319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 240/476] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:45.492818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753290e-4d6a-4ad8-90a2-e0c2676599e3 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:45.594132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950928b-c7c7-4337-bdbb-7f7eec7d43ca None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:45.636199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a18c8-9e53-4ce9-bc3e-d8871361e2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:45.636862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a18c8-9e53-4ce9-bc3e-d8871361e2ae None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:45.637155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a18c8-9e53-4ce9-bc3e-d8871361e2ae None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:45.637467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a18c8-9e53-4ce9-bc3e-d8871361e2ae None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:45.638236 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a18c8-9e53-4ce9-bc3e-d8871361e2ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:45.639691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a18c8-9e53-4ce9-bc3e-d8871361e2ae None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:45.657036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753290e-4d6a-4ad8-90a2-e0c2676599e3 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:45.672323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:45.694643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaec9f0-e7e9-4411-a9a9-c601f76ac9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:45.695471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaec9f0-e7e9-4411-a9a9-c601f76ac9ec None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:45.695760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaec9f0-e7e9-4411-a9a9-c601f76ac9ec None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:45.696035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaec9f0-e7e9-4411-a9a9-c601f76ac9ec None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:45.696787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaec9f0-e7e9-4411-a9a9-c601f76ac9ec None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:45.698299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaec9f0-e7e9-4411-a9a9-c601f76ac9ec None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:45.737199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.sql.core [None req-3eaec9f0-e7e9-4411-a9a9-c601f76ac9ec None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 18:37:45.737199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 18:37:45.737199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [parameters: {'id': 'f717d96704eb49d69e694c35fff99438', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 18:37:45.737199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92269) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-3eaec9f0-e7e9-4411-a9a9-c601f76ac9ec None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application result.read() Jun 16 18:37:45.761181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:37:45.762099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application yield resource Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise self.value Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application session.commit() Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 18:37:45.763195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self.session.flush() Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application compat.raise_( Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 18:37:45.764364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:37:45.765590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application result.read() Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:37:45.766517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [parameters: {'id': 'f717d96704eb49d69e694c35fff99438', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:45.768117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 18:37:45.773542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:45.781497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 241/477] 10.222.0.81 () {60 vars in 1131 bytes} [Thu Jun 16 18:37:45 2022] POST /identity/v3/roles => generated 144 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:45.781497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaec9f0-e7e9-4411-a9a9-c601f76ac9ec None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:45.869222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 237/478] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:37:45 2022] POST /identity/v3/users => generated 249 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:45.883911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaec9f0-e7e9-4411-a9a9-c601f76ac9ec None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:45.922413 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3da14-183a-4b60-bb13-1f15dd122345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:45.923467 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3da14-183a-4b60-bb13-1f15dd122345 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:45.923898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3da14-183a-4b60-bb13-1f15dd122345 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:45.924339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3da14-183a-4b60-bb13-1f15dd122345 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:45.935868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3da14-183a-4b60-bb13-1f15dd122345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-02d3da14-183a-4b60-bb13-1f15dd122345 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:45.944378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:45.945468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 18:37:45.946439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:45.948774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 242/479] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:37:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:45.954113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a18c8-9e53-4ce9-bc3e-d8871361e2ae None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:45.993886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:46.101895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a18c8-9e53-4ce9-bc3e-d8871361e2ae None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:46.144469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71971e21-50a1-48d8-b624-2fe0cff1ec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:46.145161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71971e21-50a1-48d8-b624-2fe0cff1ec62 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:46.145477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71971e21-50a1-48d8-b624-2fe0cff1ec62 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:46.145751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71971e21-50a1-48d8-b624-2fe0cff1ec62 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:46.146511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71971e21-50a1-48d8-b624-2fe0cff1ec62 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:46.146821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71971e21-50a1-48d8-b624-2fe0cff1ec62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:46.148274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71971e21-50a1-48d8-b624-2fe0cff1ec62 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:46.165413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 238/480] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:47.366430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309c8f8-b670-4e4a-93cd-e323eef40912 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:47.367480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309c8f8-b670-4e4a-93cd-e323eef40912 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:47.367984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309c8f8-b670-4e4a-93cd-e323eef40912 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:47.373957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 243/481] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:47.379032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf31fb-2c62-479d-b70e-bb4af80c4f1a None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:47.379454 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf31fb-2c62-479d-b70e-bb4af80c4f1a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:47.379622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf31fb-2c62-479d-b70e-bb4af80c4f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:47.403766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-09bf31fb-2c62-479d-b70e-bb4af80c4f1a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:47.404976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-09bf31fb-2c62-479d-b70e-bb4af80c4f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:47.498780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 239/482] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:47.542287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309c8f8-b670-4e4a-93cd-e323eef40912 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:47.594389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:47.668527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271b6f5-6148-44d3-8f16-ed60b670fc1a None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:47.668830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271b6f5-6148-44d3-8f16-ed60b670fc1a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:47.669076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6271b6f5-6148-44d3-8f16-ed60b670fc1a None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:47.671755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 240/483] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:37:47.675904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c13aa5-df1e-44ff-8f96-5f6a5afb92df None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:47.676197 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c13aa5-df1e-44ff-8f96-5f6a5afb92df None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:47.676481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c13aa5-df1e-44ff-8f96-5f6a5afb92df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:47.680138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309c8f8-b670-4e4a-93cd-e323eef40912 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:47.696146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-65c13aa5-df1e-44ff-8f96-5f6a5afb92df None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:47.697369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-65c13aa5-df1e-44ff-8f96-5f6a5afb92df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:47.707161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:47.751825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c114c-7f4c-42a4-b451-74d6003a425d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:47.752636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c114c-7f4c-42a4-b451-74d6003a425d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:47.753036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c114c-7f4c-42a4-b451-74d6003a425d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:47.753205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c114c-7f4c-42a4-b451-74d6003a425d None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:47.778857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c114c-7f4c-42a4-b451-74d6003a425d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-1c0c114c-7f4c-42a4-b451-74d6003a425d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:47.787049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:47.787920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:47.788715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 244/484] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:47.790839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 241/485] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:37:47.795511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c114c-7f4c-42a4-b451-74d6003a425d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:47.853522 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c13aa5-df1e-44ff-8f96-5f6a5afb92df None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:47.951278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:47.962351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c114c-7f4c-42a4-b451-74d6003a425d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:48.025981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199378d-ad44-4b05-a55f-5df98dfc4709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:48.026534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c13aa5-df1e-44ff-8f96-5f6a5afb92df None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:48.026706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199378d-ad44-4b05-a55f-5df98dfc4709 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.026955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199378d-ad44-4b05-a55f-5df98dfc4709 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.027235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199378d-ad44-4b05-a55f-5df98dfc4709 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.027986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199378d-ad44-4b05-a55f-5df98dfc4709 None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:48.028297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199378d-ad44-4b05-a55f-5df98dfc4709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:48.029795 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199378d-ad44-4b05-a55f-5df98dfc4709 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:48.047757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 245/486] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:48.064698 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199378d-ad44-4b05-a55f-5df98dfc4709 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:48.068102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b46b73-93c5-463b-8f1b-a12a450ba238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:48.068897 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b46b73-93c5-463b-8f1b-a12a450ba238 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.069181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b46b73-93c5-463b-8f1b-a12a450ba238 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.069492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b46b73-93c5-463b-8f1b-a12a450ba238 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.081641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b46b73-93c5-463b-8f1b-a12a450ba238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-e1b46b73-93c5-463b-8f1b-a12a450ba238 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:48.089813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:48.090686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:48.091456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:48.095889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 242/487] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:48.097070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b46b73-93c5-463b-8f1b-a12a450ba238 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:48.099871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:48.209320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b46b73-93c5-463b-8f1b-a12a450ba238 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:48.245595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c7040-a9ef-48b9-b2ef-76ecd62bde16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:48.246363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c7040-a9ef-48b9-b2ef-76ecd62bde16 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.246658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c7040-a9ef-48b9-b2ef-76ecd62bde16 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.246939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c7040-a9ef-48b9-b2ef-76ecd62bde16 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.247968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c7040-a9ef-48b9-b2ef-76ecd62bde16 None admin] RBAC: Adding query filter params (name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:48.248285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c7040-a9ef-48b9-b2ef-76ecd62bde16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:48.249716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c7040-a9ef-48b9-b2ef-76ecd62bde16 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:48.263881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 243/488] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:37:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:48.265412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b2ce2-a09c-44f7-9700-b6d994d0419d None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.265711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b2ce2-a09c-44f7-9700-b6d994d0419d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.265995 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b2ce2-a09c-44f7-9700-b6d994d0419d None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.267655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 244/489] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:48.268942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b2ce2-a09c-44f7-9700-b6d994d0419d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:48.283228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199378d-ad44-4b05-a55f-5df98dfc4709 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:48.317879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08721fd9-9f9a-43ea-a272-906c7ec0f8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:48.318772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08721fd9-9f9a-43ea-a272-906c7ec0f8b2 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.319059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08721fd9-9f9a-43ea-a272-906c7ec0f8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.319335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08721fd9-9f9a-43ea-a272-906c7ec0f8b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.330574 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08721fd9-9f9a-43ea-a272-906c7ec0f8b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:48.332038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08721fd9-9f9a-43ea-a272-906c7ec0f8b2 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:48.340411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-08721fd9-9f9a-43ea-a272-906c7ec0f8b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:48.354501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:48.356449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:48.357329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:48.357329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:48.357329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:48.357329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:48.357329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 246/490] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:48.357329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2e3b7-c3e7-459c-a879-138026d3c21d None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.357671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2e3b7-c3e7-459c-a879-138026d3c21d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.358168 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2e3b7-c3e7-459c-a879-138026d3c21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.382011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-71e2e3b7-c3e7-459c-a879-138026d3c21d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:48.383949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-71e2e3b7-c3e7-459c-a879-138026d3c21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:48.388268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:48.390680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b2ce2-a09c-44f7-9700-b6d994d0419d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:48.427541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f7f8d-964b-43ca-a997-0a5b602b34fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:48.429608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7f8d-964b-43ca-a997-0a5b602b34fa None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.429894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7f8d-964b-43ca-a997-0a5b602b34fa None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.430172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807f7f8d-964b-43ca-a997-0a5b602b34fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.442023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f7f8d-964b-43ca-a997-0a5b602b34fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:48.443661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807f7f8d-964b-43ca-a997-0a5b602b34fa None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-807f7f8d-964b-43ca-a997-0a5b602b34fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:48.463739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:48.464800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:48.465776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:48.465776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:48.465776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:48.465776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:48.466047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 247/492] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:48.467126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2e3b7-c3e7-459c-a879-138026d3c21d None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:48.469408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 245/492] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:48.471079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f7f8d-964b-43ca-a997-0a5b602b34fa None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:48.582916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807f7f8d-964b-43ca-a997-0a5b602b34fa None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:48.591997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2e3b7-c3e7-459c-a879-138026d3c21d None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:48.599574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:48.617635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e590a42-3a6f-4ba3-8c68-45336be733fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:48.618299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e590a42-3a6f-4ba3-8c68-45336be733fa None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.618582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e590a42-3a6f-4ba3-8c68-45336be733fa None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.619508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e590a42-3a6f-4ba3-8c68-45336be733fa None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.619598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e590a42-3a6f-4ba3-8c68-45336be733fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:48.619868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e590a42-3a6f-4ba3-8c68-45336be733fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:48.621269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e590a42-3a6f-4ba3-8c68-45336be733fa None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:48.628282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7c01c-9f72-426b-8f13-883d76ee35d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:48.628950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7c01c-9f72-426b-8f13-883d76ee35d5 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.629268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7c01c-9f72-426b-8f13-883d76ee35d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.629546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7c01c-9f72-426b-8f13-883d76ee35d5 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.630257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7c01c-9f72-426b-8f13-883d76ee35d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:48.630567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7c01c-9f72-426b-8f13-883d76ee35d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:48.631989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7c01c-9f72-426b-8f13-883d76ee35d5 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:48.637771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 246/493] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:48.637771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e590a42-3a6f-4ba3-8c68-45336be733fa None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:48.646165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:48.654289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 248/494] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:48.654289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d021527-f220-4b42-a5ca-0fec2e22bd6a None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.654289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d021527-f220-4b42-a5ca-0fec2e22bd6a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.654289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d021527-f220-4b42-a5ca-0fec2e22bd6a None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.654289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 249/495] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:48.655060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d021527-f220-4b42-a5ca-0fec2e22bd6a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:48.747600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e590a42-3a6f-4ba3-8c68-45336be733fa None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:48.759880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d021527-f220-4b42-a5ca-0fec2e22bd6a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:48.783983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04180c56-aaa6-4505-b9dd-e9107bee6191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:48.784800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04180c56-aaa6-4505-b9dd-e9107bee6191 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.785085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04180c56-aaa6-4505-b9dd-e9107bee6191 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.785399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04180c56-aaa6-4505-b9dd-e9107bee6191 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.796111 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42765403-5cf9-4525-ad60-938dbc36da44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:48.796939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42765403-5cf9-4525-ad60-938dbc36da44 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.797225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42765403-5cf9-4525-ad60-938dbc36da44 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.797392 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04180c56-aaa6-4505-b9dd-e9107bee6191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:48.797584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42765403-5cf9-4525-ad60-938dbc36da44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-04180c56-aaa6-4505-b9dd-e9107bee6191 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:48.805294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:48.806268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:48.807569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 247/496] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:48.808213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04180c56-aaa6-4505-b9dd-e9107bee6191 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:48.812759 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42765403-5cf9-4525-ad60-938dbc36da44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:48.814290 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42765403-5cf9-4525-ad60-938dbc36da44 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-42765403-5cf9-4525-ad60-938dbc36da44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:37:48.831399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:48.832323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:48.833323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:48.833323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:48.833323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:37:48.833323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:37:48.833323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:37:48.833323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:37:48.833323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:37:48.833323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:37:48.833323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:37:48.833323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:48.833323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 250/497] 10.222.0.81 () {56 vars in 1093 bytes} [Thu Jun 16 18:37:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:48.833899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e398c-1323-43a5-941e-ffe567fff280 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.834492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e398c-1323-43a5-941e-ffe567fff280 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.834492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4e398c-1323-43a5-941e-ffe567fff280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.850091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:48.850954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1a4e398c-1323-43a5-941e-ffe567fff280 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:48.854296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1a4e398c-1323-43a5-941e-ffe567fff280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:48.908783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:48.915536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04180c56-aaa6-4505-b9dd-e9107bee6191 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:48.919940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 251/498] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:48.920741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e398c-1323-43a5-941e-ffe567fff280 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:48.959657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc3870-9ec2-45d3-923b-316176da54ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:48.960438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc3870-9ec2-45d3-923b-316176da54ad None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:48.960729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc3870-9ec2-45d3-923b-316176da54ad None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:48.961002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc3870-9ec2-45d3-923b-316176da54ad None admin] PATH_INFO: `/v3/users/glance` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:48.968048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc3870-9ec2-45d3-923b-316176da54ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:48.969581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc3870-9ec2-45d3-923b-316176da54ad None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-c5bc3870-9ec2-45d3-923b-316176da54ad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:37:48.984764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:48.985716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:48.986592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:48.986592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:48.986592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:37:48.986592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:37:48.986592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:37:48.986592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:37:48.986592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:37:48.986592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:37:48.986592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:37:48.986592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:48.986592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 248/499] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:48.987141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc3870-9ec2-45d3-923b-316176da54ad None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:49.023449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4e398c-1323-43a5-941e-ffe567fff280 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:49.063875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd0402a-20dd-43e3-aaec-08588e91baa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:49.064545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0402a-20dd-43e3-aaec-08588e91baa9 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.064829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0402a-20dd-43e3-aaec-08588e91baa9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.065107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0402a-20dd-43e3-aaec-08588e91baa9 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.065943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0402a-20dd-43e3-aaec-08588e91baa9 None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:49.066254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0402a-20dd-43e3-aaec-08588e91baa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:49.067663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd0402a-20dd-43e3-aaec-08588e91baa9 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:49.080063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 252/500] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:37:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:49.080063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012973-2ed5-43f1-8506-2fb8b31b0d85 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.080063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012973-2ed5-43f1-8506-2fb8b31b0d85 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.080063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b012973-2ed5-43f1-8506-2fb8b31b0d85 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.080063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 253/501] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:49.080063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b012973-2ed5-43f1-8506-2fb8b31b0d85 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:49.087140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:49.092653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc3870-9ec2-45d3-923b-316176da54ad None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:49.126317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85edda-f914-46c1-b52e-06f7dc545bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:49.127288 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85edda-f914-46c1-b52e-06f7dc545bc9 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.127617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85edda-f914-46c1-b52e-06f7dc545bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.127892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85edda-f914-46c1-b52e-06f7dc545bc9 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.128751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85edda-f914-46c1-b52e-06f7dc545bc9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:49.129058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85edda-f914-46c1-b52e-06f7dc545bc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:49.130544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85edda-f914-46c1-b52e-06f7dc545bc9 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:49.133485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:49.167273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 249/502] 10.222.0.81 () {56 vars in 1133 bytes} [Thu Jun 16 18:37:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:49.168000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85edda-f914-46c1-b52e-06f7dc545bc9 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:49.184860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b012973-2ed5-43f1-8506-2fb8b31b0d85 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:49.217023 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c0d68-1963-4ad0-a350-89ab65b7cb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:49.217850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c0d68-1963-4ad0-a350-89ab65b7cb9c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.218116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c0d68-1963-4ad0-a350-89ab65b7cb9c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.218386 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c0d68-1963-4ad0-a350-89ab65b7cb9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.228033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c0d68-1963-4ad0-a350-89ab65b7cb9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-731c0d68-1963-4ad0-a350-89ab65b7cb9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:49.236094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:49.237147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 254/503] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:49.238116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c0d68-1963-4ad0-a350-89ab65b7cb9c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:49.265857 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85edda-f914-46c1-b52e-06f7dc545bc9 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:49.299093 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b4d77-619a-46b1-9635-803e8fd1262f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:49.299726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b4d77-619a-46b1-9635-803e8fd1262f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.300006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b4d77-619a-46b1-9635-803e8fd1262f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.300292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b4d77-619a-46b1-9635-803e8fd1262f None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.301114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b4d77-619a-46b1-9635-803e8fd1262f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:49.301453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b4d77-619a-46b1-9635-803e8fd1262f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:49.302899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b4d77-619a-46b1-9635-803e8fd1262f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:49.319726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 250/504] 10.222.0.81 () {56 vars in 1137 bytes} [Thu Jun 16 18:37:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:49.320910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e7c9-3241-4f55-ae0c-c949a99d893e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.321198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e7c9-3241-4f55-ae0c-c949a99d893e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.321506 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e7c9-3241-4f55-ae0c-c949a99d893e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.336700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c0d68-1963-4ad0-a350-89ab65b7cb9c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:49.337426 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-2989e7c9-3241-4f55-ae0c-c949a99d893e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:49.338620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-2989e7c9-3241-4f55-ae0c-c949a99d893e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:49.346436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:49.351310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:49.372905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3903f7c-bfab-4cb8-8a48-7a6802729663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:49.373733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3903f7c-bfab-4cb8-8a48-7a6802729663 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.374015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3903f7c-bfab-4cb8-8a48-7a6802729663 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.374285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3903f7c-bfab-4cb8-8a48-7a6802729663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.384665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3903f7c-bfab-4cb8-8a48-7a6802729663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:49.386299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3903f7c-bfab-4cb8-8a48-7a6802729663 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-a3903f7c-bfab-4cb8-8a48-7a6802729663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:49.403085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:49.403931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:49.404816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:49.404816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:49.404816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:49.404816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:49.404816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 255/505] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:49.405476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3903f7c-bfab-4cb8-8a48-7a6802729663 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:49.410300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 251/506] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:49.411138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989e7c9-3241-4f55-ae0c-c949a99d893e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:49.506263 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3903f7c-bfab-4cb8-8a48-7a6802729663 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:49.508217 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989e7c9-3241-4f55-ae0c-c949a99d893e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:49.539369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccf863-bc0b-4f5b-932a-1b47c1d38501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:49.540002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccf863-bc0b-4f5b-932a-1b47c1d38501 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.540281 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccf863-bc0b-4f5b-932a-1b47c1d38501 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.540554 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccf863-bc0b-4f5b-932a-1b47c1d38501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.540834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948a41c-1066-4ea1-a436-288d2f7f5801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:49.542121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948a41c-1066-4ea1-a436-288d2f7f5801 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.542403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948a41c-1066-4ea1-a436-288d2f7f5801 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.542676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948a41c-1066-4ea1-a436-288d2f7f5801 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.543412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948a41c-1066-4ea1-a436-288d2f7f5801 None admin] RBAC: Adding query filter params (name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:49.543892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948a41c-1066-4ea1-a436-288d2f7f5801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:49.545290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948a41c-1066-4ea1-a436-288d2f7f5801 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:49.553095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 252/507] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:37:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:49.553838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948a41c-1066-4ea1-a436-288d2f7f5801 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:49.557210 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:49.577562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:49.614202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccf863-bc0b-4f5b-932a-1b47c1d38501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:49.615677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccf863-bc0b-4f5b-932a-1b47c1d38501 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:49.652694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948a41c-1066-4ea1-a436-288d2f7f5801 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:49.685683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68879f-c924-45b4-b1e8-deb26df2917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:49.686325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68879f-c924-45b4-b1e8-deb26df2917f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.686612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68879f-c924-45b4-b1e8-deb26df2917f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.686886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68879f-c924-45b4-b1e8-deb26df2917f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.711989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 256/508] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:49.712867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccf863-bc0b-4f5b-932a-1b47c1d38501 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:49.753936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68879f-c924-45b4-b1e8-deb26df2917f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:49.755384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68879f-c924-45b4-b1e8-deb26df2917f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:49.758080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:49.771478 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:49.807764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccf863-bc0b-4f5b-932a-1b47c1d38501 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:49.841088 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285d570-ff21-4b92-9753-8b13de2fff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:49.841784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285d570-ff21-4b92-9753-8b13de2fff98 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.842072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285d570-ff21-4b92-9753-8b13de2fff98 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.842347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285d570-ff21-4b92-9753-8b13de2fff98 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.843053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285d570-ff21-4b92-9753-8b13de2fff98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:49.843360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285d570-ff21-4b92-9753-8b13de2fff98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:49.844723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285d570-ff21-4b92-9753-8b13de2fff98 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:49.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 253/509] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:49.851314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68879f-c924-45b4-b1e8-deb26df2917f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:49.857204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 257/510] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:49.857917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285d570-ff21-4b92-9753-8b13de2fff98 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:49.950535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68879f-c924-45b4-b1e8-deb26df2917f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:49.952879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:49.954178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285d570-ff21-4b92-9753-8b13de2fff98 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:49.976708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:49.983734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ae3a8-4b4b-4971-a1d6-3b046ebc45ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:49.985048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ae3a8-4b4b-4971-a1d6-3b046ebc45ee None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.985362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ae3a8-4b4b-4971-a1d6-3b046ebc45ee None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.985631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ae3a8-4b4b-4971-a1d6-3b046ebc45ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.987585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e200017-4e48-4ddf-9378-80e0db22fa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:49.988367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e200017-4e48-4ddf-9378-80e0db22fa64 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:49.988704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e200017-4e48-4ddf-9378-80e0db22fa64 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:49.988923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e200017-4e48-4ddf-9378-80e0db22fa64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:49.996324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ae3a8-4b4b-4971-a1d6-3b046ebc45ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:49.997813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ae3a8-4b4b-4971-a1d6-3b046ebc45ee None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:49.999292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e200017-4e48-4ddf-9378-80e0db22fa64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:50.000713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e200017-4e48-4ddf-9378-80e0db22fa64 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-396ae3a8-4b4b-4971-a1d6-3b046ebc45ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:50.015000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:50.015974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:50.016808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:50.016808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:50.016808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:50.016808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:50.016808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 254/511] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:50.017359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ae3a8-4b4b-4971-a1d6-3b046ebc45ee None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-0e200017-4e48-4ddf-9378-80e0db22fa64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:50.017745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:50.018712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:50.019612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:50.019612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:50.019612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:50.019612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:50.019612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 258/512] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:50.020060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e200017-4e48-4ddf-9378-80e0db22fa64 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.116640 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e200017-4e48-4ddf-9378-80e0db22fa64 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:50.124064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ae3a8-4b4b-4971-a1d6-3b046ebc45ee None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:50.151586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9090f-e732-4e84-aa9e-ad486a39570f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:50.152365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9090f-e732-4e84-aa9e-ad486a39570f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:50.152644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9090f-e732-4e84-aa9e-ad486a39570f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:50.152912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9090f-e732-4e84-aa9e-ad486a39570f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:50.156766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc382dc4-bd9b-4de5-86c4-f00df220caa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:50.157579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc382dc4-bd9b-4de5-86c4-f00df220caa6 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:50.157868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc382dc4-bd9b-4de5-86c4-f00df220caa6 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:50.158147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc382dc4-bd9b-4de5-86c4-f00df220caa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:50.162466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9090f-e732-4e84-aa9e-ad486a39570f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:50.163908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9090f-e732-4e84-aa9e-ad486a39570f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:50.169745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc382dc4-bd9b-4de5-86c4-f00df220caa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:50.171188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc382dc4-bd9b-4de5-86c4-f00df220caa6 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-31d9090f-e732-4e84-aa9e-ad486a39570f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:50.181191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:50.182271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:50.183298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:50.183298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:50.183298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:50.183298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:50.183298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 259/513] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:50.183733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9090f-e732-4e84-aa9e-ad486a39570f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-bc382dc4-bd9b-4de5-86c4-f00df220caa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:50.190817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:50.191658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:50.193598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:50.193598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:50.193598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:50.193598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:50.193598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 255/514] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:50.193598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc382dc4-bd9b-4de5-86c4-f00df220caa6 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.203711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:50.210798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:50.281344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9090f-e732-4e84-aa9e-ad486a39570f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:50.289932 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc382dc4-bd9b-4de5-86c4-f00df220caa6 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:50.314248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83818de-9887-4b39-9a86-20eb27e7f3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:50.315042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83818de-9887-4b39-9a86-20eb27e7f3cb None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:50.315333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83818de-9887-4b39-9a86-20eb27e7f3cb None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:50.315610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83818de-9887-4b39-9a86-20eb27e7f3cb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:50.322455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83818de-9887-4b39-9a86-20eb27e7f3cb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:50.322588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4ed2b-d80a-4aa2-80ab-2e892d7d4d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:50.323133 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4ed2b-d80a-4aa2-80ab-2e892d7d4d8e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:50.323417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4ed2b-d80a-4aa2-80ab-2e892d7d4d8e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:50.323695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4ed2b-d80a-4aa2-80ab-2e892d7d4d8e None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:50.323996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83818de-9887-4b39-9a86-20eb27e7f3cb None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:50.324403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4ed2b-d80a-4aa2-80ab-2e892d7d4d8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:50.324710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4ed2b-d80a-4aa2-80ab-2e892d7d4d8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:50.326098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4ed2b-d80a-4aa2-80ab-2e892d7d4d8e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:50.337438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 256/515] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-c83818de-9887-4b39-9a86-20eb27e7f3cb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:37:50.338045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:50.339423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:50.340309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:50.340309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:50.340309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:37:50.340309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:37:50.340309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:37:50.340309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:37:50.340309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:37:50.340309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:37:50.340309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:37:50.340309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:50.340817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4ed2b-d80a-4aa2-80ab-2e892d7d4d8e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.340885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 260/516] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:50.340885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83818de-9887-4b39-9a86-20eb27e7f3cb None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.409036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:50.430118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:50.442964 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83818de-9887-4b39-9a86-20eb27e7f3cb None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:50.443265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4ed2b-d80a-4aa2-80ab-2e892d7d4d8e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:50.476798 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7f270-f544-490b-a797-5c5144f53cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:50.477150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59554e3-748e-47f2-8b62-9dacfba99232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:50.477488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7f270-f544-490b-a797-5c5144f53cb8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:50.477774 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7f270-f544-490b-a797-5c5144f53cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:50.477865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59554e3-748e-47f2-8b62-9dacfba99232 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:50.478051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d7f270-f544-490b-a797-5c5144f53cb8 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:50.478149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59554e3-748e-47f2-8b62-9dacfba99232 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:50.478371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59554e3-748e-47f2-8b62-9dacfba99232 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:50.478769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7f270-f544-490b-a797-5c5144f53cb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:50.479108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59554e3-748e-47f2-8b62-9dacfba99232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:50.479205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7f270-f544-490b-a797-5c5144f53cb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:50.479417 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59554e3-748e-47f2-8b62-9dacfba99232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:50.480494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d7f270-f544-490b-a797-5c5144f53cb8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:50.480775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59554e3-748e-47f2-8b62-9dacfba99232 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:50.491834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 261/517] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:50.497400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 257/518] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:50.497543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59554e3-748e-47f2-8b62-9dacfba99232 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.497654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7f270-f544-490b-a797-5c5144f53cb8 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.592038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59554e3-748e-47f2-8b62-9dacfba99232 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:50.592229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d7f270-f544-490b-a797-5c5144f53cb8 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:50.623387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:50.625426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1554cd-91c6-414d-bd25-7ae57228aad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:50.626089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1554cd-91c6-414d-bd25-7ae57228aad2 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:50.626380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1554cd-91c6-414d-bd25-7ae57228aad2 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:50.626649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1554cd-91c6-414d-bd25-7ae57228aad2 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:50.627348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1554cd-91c6-414d-bd25-7ae57228aad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:50.627651 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1554cd-91c6-414d-bd25-7ae57228aad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:50.629014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1554cd-91c6-414d-bd25-7ae57228aad2 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:50.631053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c00670-21c3-42f7-9aa4-9fc5d5854004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:50.631722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c00670-21c3-42f7-9aa4-9fc5d5854004 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:50.632007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c00670-21c3-42f7-9aa4-9fc5d5854004 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:50.632286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c00670-21c3-42f7-9aa4-9fc5d5854004 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:50.632983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c00670-21c3-42f7-9aa4-9fc5d5854004 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:50.634463 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c00670-21c3-42f7-9aa4-9fc5d5854004 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:50.640967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 262/519] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:50.641824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1554cd-91c6-414d-bd25-7ae57228aad2 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.651942 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:50.705765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 258/520] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:37:50 2022] POST /identity/v3/users => generated 247 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:37:50.705765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c00670-21c3-42f7-9aa4-9fc5d5854004 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.746132 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1554cd-91c6-414d-bd25-7ae57228aad2 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:50.800611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356ebd5-a820-4d1f-b798-e255a859c234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:50.801353 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356ebd5-a820-4d1f-b798-e255a859c234 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:50.801641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356ebd5-a820-4d1f-b798-e255a859c234 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:50.801918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356ebd5-a820-4d1f-b798-e255a859c234 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:50.802759 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356ebd5-a820-4d1f-b798-e255a859c234 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:50.803073 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356ebd5-a820-4d1f-b798-e255a859c234 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:50.804562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356ebd5-a820-4d1f-b798-e255a859c234 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:50.817359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c00670-21c3-42f7-9aa4-9fc5d5854004 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:50.846646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 263/521] 10.222.0.81 () {56 vars in 1137 bytes} [Thu Jun 16 18:37:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:50.847528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356ebd5-a820-4d1f-b798-e255a859c234 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.863555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22ce43-0863-424f-b7fe-73bd97ac0430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:50.864661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22ce43-0863-424f-b7fe-73bd97ac0430 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:50.864943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22ce43-0863-424f-b7fe-73bd97ac0430 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:50.865215 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22ce43-0863-424f-b7fe-73bd97ac0430 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:50.873055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22ce43-0863-424f-b7fe-73bd97ac0430 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:50.874568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22ce43-0863-424f-b7fe-73bd97ac0430 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-8f22ce43-0863-424f-b7fe-73bd97ac0430 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:37:50.891292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:50.892188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:50.893141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:37:50.893141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:37:50.893141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:37:50.893141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:37:50.893141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:37:50.893141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:37:50.893141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:50.893141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:50.893141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 259/522] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:50.894190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22ce43-0863-424f-b7fe-73bd97ac0430 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:50.924761 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:50.952771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:50.961039 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356ebd5-a820-4d1f-b798-e255a859c234 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:50.997884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3a22-53d2-4aac-9ba8-dcbfc5ca3b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:50.998803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3a22-53d2-4aac-9ba8-dcbfc5ca3b7c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:50.999100 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3a22-53d2-4aac-9ba8-dcbfc5ca3b7c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:50.999361 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe3a22-53d2-4aac-9ba8-dcbfc5ca3b7c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:51.007372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe3a22-53d2-4aac-9ba8-dcbfc5ca3b7c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:51.008890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe3a22-53d2-4aac-9ba8-dcbfc5ca3b7c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:51.015605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22ce43-0863-424f-b7fe-73bd97ac0430 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-37fe3a22-53d2-4aac-9ba8-dcbfc5ca3b7c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:37:51.024278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:51.025405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 264/523] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:51.026531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3a22-53d2-4aac-9ba8-dcbfc5ca3b7c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:51.063602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201423f-89a9-4687-8e18-0549f740bc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:51.064686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201423f-89a9-4687-8e18-0549f740bc55 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:51.065126 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201423f-89a9-4687-8e18-0549f740bc55 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:51.065558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201423f-89a9-4687-8e18-0549f740bc55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:51.076491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201423f-89a9-4687-8e18-0549f740bc55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:51.078147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201423f-89a9-4687-8e18-0549f740bc55 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-5201423f-89a9-4687-8e18-0549f740bc55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:37:51.094245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:51.095304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:51.096330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:37:51.096330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:37:51.096330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:37:51.096330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:37:51.096330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:37:51.096330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:37:51.096330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:51.096330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:51.098216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 260/524] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:51.099128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201423f-89a9-4687-8e18-0549f740bc55 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:51.156164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe3a22-53d2-4aac-9ba8-dcbfc5ca3b7c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:51.186987 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:51.196616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08c008-5786-4e88-bab0-88be97b23a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:51.197295 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08c008-5786-4e88-bab0-88be97b23a61 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:51.197579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08c008-5786-4e88-bab0-88be97b23a61 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:51.201090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08c008-5786-4e88-bab0-88be97b23a61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:51.213230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201423f-89a9-4687-8e18-0549f740bc55 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:51.216391 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:51.261701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28437ea-a8bb-4da9-98f4-bdf7f587adff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:51.262890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28437ea-a8bb-4da9-98f4-bdf7f587adff None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:51.263367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28437ea-a8bb-4da9-98f4-bdf7f587adff None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:51.263822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28437ea-a8bb-4da9-98f4-bdf7f587adff None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:51.264980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28437ea-a8bb-4da9-98f4-bdf7f587adff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:51.265320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28437ea-a8bb-4da9-98f4-bdf7f587adff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:51.266758 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28437ea-a8bb-4da9-98f4-bdf7f587adff None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:51.274571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08c008-5786-4e88-bab0-88be97b23a61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:51.276148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08c008-5786-4e88-bab0-88be97b23a61 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:51.289151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 261/525] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:37:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:51.289934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28437ea-a8bb-4da9-98f4-bdf7f587adff None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:51.390988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 265/526] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:51.392571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08c008-5786-4e88-bab0-88be97b23a61 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:51.417001 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28437ea-a8bb-4da9-98f4-bdf7f587adff None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:51.423345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:51.458594 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6eebb-2694-48db-8a55-b7c806aa80ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:51.459367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6eebb-2694-48db-8a55-b7c806aa80ba None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:51.459649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6eebb-2694-48db-8a55-b7c806aa80ba None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:51.461915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6eebb-2694-48db-8a55-b7c806aa80ba None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:51.461915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6eebb-2694-48db-8a55-b7c806aa80ba None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:51.461915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6eebb-2694-48db-8a55-b7c806aa80ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:51.462569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6eebb-2694-48db-8a55-b7c806aa80ba None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:51.485839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 262/527] 10.222.0.81 () {56 vars in 1139 bytes} [Thu Jun 16 18:37:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:51.486601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6eebb-2694-48db-8a55-b7c806aa80ba None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:51.516306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:51.520911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08c008-5786-4e88-bab0-88be97b23a61 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:51.558615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910a3b4-c01b-4171-84c7-0c2cc3215619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:51.559436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910a3b4-c01b-4171-84c7-0c2cc3215619 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:51.560151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910a3b4-c01b-4171-84c7-0c2cc3215619 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:51.560151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910a3b4-c01b-4171-84c7-0c2cc3215619 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:51.560765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910a3b4-c01b-4171-84c7-0c2cc3215619 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:51.561078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910a3b4-c01b-4171-84c7-0c2cc3215619 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:51.562590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910a3b4-c01b-4171-84c7-0c2cc3215619 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:51.582109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 266/528] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:37:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:51.587628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910a3b4-c01b-4171-84c7-0c2cc3215619 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:51.594522 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6eebb-2694-48db-8a55-b7c806aa80ba None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:51.628535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3fb73-2698-46f1-bc4d-c8970a9c0994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:51.629184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3fb73-2698-46f1-bc4d-c8970a9c0994 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:51.629527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3fb73-2698-46f1-bc4d-c8970a9c0994 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:51.629794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e3fb73-2698-46f1-bc4d-c8970a9c0994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:51.630622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3fb73-2698-46f1-bc4d-c8970a9c0994 None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=d17b7a7128d64b2ba548add02d3ec264) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:51.630936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3fb73-2698-46f1-bc4d-c8970a9c0994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:51.632373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e3fb73-2698-46f1-bc4d-c8970a9c0994 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:51.638513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 263/529] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:51 2022] GET /identity/v3/role_assignments?user.id=d17b7a7128d64b2ba548add02d3ec264&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:51.639279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3fb73-2698-46f1-bc4d-c8970a9c0994 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:51.691601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:51.709006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910a3b4-c01b-4171-84c7-0c2cc3215619 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:51.745683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:51.750689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaee44f-64cd-4315-a9fb-0147c9ddaab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:51.751508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaee44f-64cd-4315-a9fb-0147c9ddaab7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:51.751792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaee44f-64cd-4315-a9fb-0147c9ddaab7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:51.752065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaee44f-64cd-4315-a9fb-0147c9ddaab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:51.764947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e3fb73-2698-46f1-bc4d-c8970a9c0994 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:51.765192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaee44f-64cd-4315-a9fb-0147c9ddaab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:51.766541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaee44f-64cd-4315-a9fb-0147c9ddaab7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-bdaee44f-64cd-4315-a9fb-0147c9ddaab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:51.792284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:51.793687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:51.794524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:51.794524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:51.794524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:51.794524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:51.794524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 267/530] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:51.794797 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaee44f-64cd-4315-a9fb-0147c9ddaab7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:51.807916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9f38e-cec7-4098-9c06-a3d307e414f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:51.809015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9f38e-cec7-4098-9c06-a3d307e414f6 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:51.809558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9f38e-cec7-4098-9c06-a3d307e414f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:51.809906 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9f38e-cec7-4098-9c06-a3d307e414f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:51.897120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9f38e-cec7-4098-9c06-a3d307e414f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:51.898756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9f38e-cec7-4098-9c06-a3d307e414f6 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:51.931638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaee44f-64cd-4315-a9fb-0147c9ddaab7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:51.978274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df963e-a876-49e0-a861-71c75ac4de36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:51.979302 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df963e-a876-49e0-a861-71c75ac4de36 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:51.980273 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df963e-a876-49e0-a861-71c75ac4de36 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:51.980727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df963e-a876-49e0-a861-71c75ac4de36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:51.981794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df963e-a876-49e0-a861-71c75ac4de36 None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=251ed026ce8046e98188066bf0bc40b7) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:51.982819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df963e-a876-49e0-a861-71c75ac4de36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:51.984906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df963e-a876-49e0-a861-71c75ac4de36 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:51.993560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:51.994639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 268/531] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:51 2022] GET /identity/v3/role_assignments?user.id=251ed026ce8046e98188066bf0bc40b7&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:51.996276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df963e-a876-49e0-a861-71c75ac4de36 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:52.000130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:52.014000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 264/532] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:52.021107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9f38e-cec7-4098-9c06-a3d307e414f6 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:52.135741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9f38e-cec7-4098-9c06-a3d307e414f6 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:52.151336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df963e-a876-49e0-a861-71c75ac4de36 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:52.178713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09f58d-4c06-4666-9d61-83f39ac0eac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:52.179528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f58d-4c06-4666-9d61-83f39ac0eac2 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:52.179806 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f58d-4c06-4666-9d61-83f39ac0eac2 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:52.180076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f58d-4c06-4666-9d61-83f39ac0eac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:52.201094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09f58d-4c06-4666-9d61-83f39ac0eac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:52.202625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09f58d-4c06-4666-9d61-83f39ac0eac2 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:52.212143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d01447-ce59-490b-8745-4b264283b81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:52.212884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d01447-ce59-490b-8745-4b264283b81d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:52.213170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d01447-ce59-490b-8745-4b264283b81d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:52.213491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d01447-ce59-490b-8745-4b264283b81d None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:52.214221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d01447-ce59-490b-8745-4b264283b81d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:52.214527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d01447-ce59-490b-8745-4b264283b81d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:52.216707 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d01447-ce59-490b-8745-4b264283b81d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-ab09f58d-4c06-4666-9d61-83f39ac0eac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:52.237466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:52.238417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:52.239257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:52.239257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:52.239257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:52.239257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:52.240857 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 265/533] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:52.242870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 269/534] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:52.246699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09f58d-4c06-4666-9d61-83f39ac0eac2 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:52.248485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d01447-ce59-490b-8745-4b264283b81d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:52.302342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:52.381088 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09f58d-4c06-4666-9d61-83f39ac0eac2 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:52.390991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:52.445316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7310bf-0a23-4a67-b00d-01ac9c92e9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:52.445990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7310bf-0a23-4a67-b00d-01ac9c92e9c1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:52.446274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7310bf-0a23-4a67-b00d-01ac9c92e9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:52.446548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7310bf-0a23-4a67-b00d-01ac9c92e9c1 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:52.447273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7310bf-0a23-4a67-b00d-01ac9c92e9c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:52.447586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7310bf-0a23-4a67-b00d-01ac9c92e9c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:52.449014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7310bf-0a23-4a67-b00d-01ac9c92e9c1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:52.461519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 266/535] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:52.467428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7310bf-0a23-4a67-b00d-01ac9c92e9c1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:52.475359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d01447-ce59-490b-8745-4b264283b81d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:52.562486 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5a895-3703-417e-bf5c-db8b79332ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:52.563293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5a895-3703-417e-bf5c-db8b79332ae4 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:52.563576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5a895-3703-417e-bf5c-db8b79332ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:52.563849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5a895-3703-417e-bf5c-db8b79332ae4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:52.569647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:52.573665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7310bf-0a23-4a67-b00d-01ac9c92e9c1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:52.582379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5a895-3703-417e-bf5c-db8b79332ae4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:52.583858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5a895-3703-417e-bf5c-db8b79332ae4 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-bab5a895-3703-417e-bf5c-db8b79332ae4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:37:52.607376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:52.608342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:52.609181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:37:52.609181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:37:52.609181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:37:52.609181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:37:52.609181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:37:52.609181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:37:52.609181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:52.609181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:52.609181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 270/536] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:52.612003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b0250-38f7-49a7-84de-387e28a31d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:52.612815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b0250-38f7-49a7-84de-387e28a31d0a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:52.613211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b0250-38f7-49a7-84de-387e28a31d0a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:52.613497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b0250-38f7-49a7-84de-387e28a31d0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:52.614883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5a895-3703-417e-bf5c-db8b79332ae4 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:52.630980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b0250-38f7-49a7-84de-387e28a31d0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:52.632454 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b0250-38f7-49a7-84de-387e28a31d0a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-898b0250-38f7-49a7-84de-387e28a31d0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:37:52.655084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:52.655969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:52.657268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:37:52.657268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:37:52.657268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:37:52.657268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:37:52.657268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:37:52.657268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:37:52.657268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:52.657268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:52.657268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 267/537] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:52.662323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b0250-38f7-49a7-84de-387e28a31d0a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:52.777134 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b0250-38f7-49a7-84de-387e28a31d0a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:52.813294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5a895-3703-417e-bf5c-db8b79332ae4 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:52.817855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f7500-9454-47af-a784-17e02ec771ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:52.817855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f7500-9454-47af-a784-17e02ec771ec None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:52.817855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f7500-9454-47af-a784-17e02ec771ec None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:52.817855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15f7500-9454-47af-a784-17e02ec771ec None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:52.818168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f7500-9454-47af-a784-17e02ec771ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:52.818168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f7500-9454-47af-a784-17e02ec771ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:52.820269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15f7500-9454-47af-a784-17e02ec771ec None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:52.834535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 268/538] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:37:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:52.840892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f7500-9454-47af-a784-17e02ec771ec None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:52.861637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:52.863145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:52.889749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c68bf-2996-456d-8990-83222529319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:52.890837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c68bf-2996-456d-8990-83222529319a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:52.891339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c68bf-2996-456d-8990-83222529319a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:52.891834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c68bf-2996-456d-8990-83222529319a None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:52.892839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c68bf-2996-456d-8990-83222529319a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:52.893371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c68bf-2996-456d-8990-83222529319a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:52.895050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c68bf-2996-456d-8990-83222529319a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:52.916015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 271/539] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:37:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:52.921678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c68bf-2996-456d-8990-83222529319a None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:52.954412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15f7500-9454-47af-a784-17e02ec771ec None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:52.991022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15094ef5-0618-4f7b-bfaa-b420230dc4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:52.991736 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094ef5-0618-4f7b-bfaa-b420230dc4e1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:52.992026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094ef5-0618-4f7b-bfaa-b420230dc4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:52.992410 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15094ef5-0618-4f7b-bfaa-b420230dc4e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:52.993546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15094ef5-0618-4f7b-bfaa-b420230dc4e1 None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=f4e05d75cd1142e18d7518b78edd88fa) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:52.993860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15094ef5-0618-4f7b-bfaa-b420230dc4e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:52.995514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15094ef5-0618-4f7b-bfaa-b420230dc4e1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:53.001820 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 269/540] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:52 2022] GET /identity/v3/role_assignments?user.id=f4e05d75cd1142e18d7518b78edd88fa&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:53.091114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c68bf-2996-456d-8990-83222529319a None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:53.147765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5dbcb-41ec-4865-863d-743cd889c12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:53.148438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5dbcb-41ec-4865-863d-743cd889c12a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:53.148718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5dbcb-41ec-4865-863d-743cd889c12a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:53.148993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5dbcb-41ec-4865-863d-743cd889c12a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:53.149957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5dbcb-41ec-4865-863d-743cd889c12a None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=9c82335b7ec74fb3a1508a2fc76335ab) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:53.150267 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5dbcb-41ec-4865-863d-743cd889c12a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:53.152314 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5dbcb-41ec-4865-863d-743cd889c12a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:53.162076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 272/541] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:52 2022] GET /identity/v3/role_assignments?user.id=9c82335b7ec74fb3a1508a2fc76335ab&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:53.187737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b3696-f223-48d6-ad74-bdb0df0d92c1 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:53.188055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b3696-f223-48d6-ad74-bdb0df0d92c1 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:53.188306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b3696-f223-48d6-ad74-bdb0df0d92c1 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:53.189999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 270/542] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:53.195902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671cea3-1bc1-4d8b-af2c-14dd82f75aec None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:53.196238 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671cea3-1bc1-4d8b-af2c-14dd82f75aec None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:53.196462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671cea3-1bc1-4d8b-af2c-14dd82f75aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:53.218107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d671cea3-1bc1-4d8b-af2c-14dd82f75aec None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:53.219343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d671cea3-1bc1-4d8b-af2c-14dd82f75aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:53.271151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:53.298797 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 273/543] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:53.339360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b3696-f223-48d6-ad74-bdb0df0d92c1 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:53.464000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:53.493781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b3696-f223-48d6-ad74-bdb0df0d92c1 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:53.543926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e77e79-d0d7-47c2-81e9-ceda1d55158c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:53.544732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e77e79-d0d7-47c2-81e9-ceda1d55158c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:53.545020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e77e79-d0d7-47c2-81e9-ceda1d55158c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:53.545319 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e77e79-d0d7-47c2-81e9-ceda1d55158c None admin] PATH_INFO: `/v3/roles/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:53.559757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e77e79-d0d7-47c2-81e9-ceda1d55158c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-56e77e79-d0d7-47c2-81e9-ceda1d55158c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:53.567427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:53.568294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:53.569540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 271/544] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:53.575054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671cea3-1bc1-4d8b-af2c-14dd82f75aec None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:53.724146 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671cea3-1bc1-4d8b-af2c-14dd82f75aec None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:53.768841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef42df-7efe-4684-9d5a-4a8cf0737eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:53.769549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef42df-7efe-4684-9d5a-4a8cf0737eb0 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:53.769835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef42df-7efe-4684-9d5a-4a8cf0737eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:53.770109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef42df-7efe-4684-9d5a-4a8cf0737eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:53.770846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef42df-7efe-4684-9d5a-4a8cf0737eb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:53.771154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef42df-7efe-4684-9d5a-4a8cf0737eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:53.772532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef42df-7efe-4684-9d5a-4a8cf0737eb0 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:53.781024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 274/545] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:37:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:53.786588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e77e79-d0d7-47c2-81e9-ceda1d55158c None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:53.921666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e77e79-d0d7-47c2-81e9-ceda1d55158c None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:53.965925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0333d-b9f0-49ee-a474-e416c37d36f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:53.966726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0333d-b9f0-49ee-a474-e416c37d36f0 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:53.967011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0333d-b9f0-49ee-a474-e416c37d36f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:53.967293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0333d-b9f0-49ee-a474-e416c37d36f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:53.983207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0333d-b9f0-49ee-a474-e416c37d36f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:53.984604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0333d-b9f0-49ee-a474-e416c37d36f0 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:53.986441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-25a0333d-b9f0-49ee-a474-e416c37d36f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:54.003882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:54.004761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:54.006285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:54.006285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:54.006285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:54.006285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:54.006285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 272/546] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:54.010903 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef42df-7efe-4684-9d5a-4a8cf0737eb0 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:54.065658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:54.142004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f4d1-30a2-4034-ae05-03381696ace6 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:54.142338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f4d1-30a2-4034-ae05-03381696ace6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:54.142559 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087f4d1-30a2-4034-ae05-03381696ace6 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:54.143150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef42df-7efe-4684-9d5a-4a8cf0737eb0 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:54.144169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 273/547] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:37:54.149973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eceebb-8e83-46ef-9e3e-fb2508e4e83d None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:54.150266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eceebb-8e83-46ef-9e3e-fb2508e4e83d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:54.150543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eceebb-8e83-46ef-9e3e-fb2508e4e83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:54.177783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-16eceebb-8e83-46ef-9e3e-fb2508e4e83d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:54.178999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-16eceebb-8e83-46ef-9e3e-fb2508e4e83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:54.210898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb839a10-df53-4a53-a6be-d9b422b9523b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:54.211571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb839a10-df53-4a53-a6be-d9b422b9523b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:54.211858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb839a10-df53-4a53-a6be-d9b422b9523b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:54.212135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb839a10-df53-4a53-a6be-d9b422b9523b None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:54.212851 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb839a10-df53-4a53-a6be-d9b422b9523b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:54.213160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb839a10-df53-4a53-a6be-d9b422b9523b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:54.215172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb839a10-df53-4a53-a6be-d9b422b9523b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:54.242208 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 275/548] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:54.247437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb839a10-df53-4a53-a6be-d9b422b9523b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:54.315207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 274/549] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:54.351185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eceebb-8e83-46ef-9e3e-fb2508e4e83d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:54.418662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:54.424440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb839a10-df53-4a53-a6be-d9b422b9523b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:54.495744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eceebb-8e83-46ef-9e3e-fb2508e4e83d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:54.525653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:54.531181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012f294-6931-419e-a13d-d6d7a478ca8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:54.532105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012f294-6931-419e-a13d-d6d7a478ca8b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:54.532382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012f294-6931-419e-a13d-d6d7a478ca8b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:54.541413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012f294-6931-419e-a13d-d6d7a478ca8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:54.541528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3eae5-3289-464c-95ff-73f66cd3eb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:54.541528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3eae5-3289-464c-95ff-73f66cd3eb18 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:54.541528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3eae5-3289-464c-95ff-73f66cd3eb18 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:54.541528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3eae5-3289-464c-95ff-73f66cd3eb18 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:54.543004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012f294-6931-419e-a13d-d6d7a478ca8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-a012f294-6931-419e-a13d-d6d7a478ca8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:54.550732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:54.551666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:54.552742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 275/550] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:54.559137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3eae5-3289-464c-95ff-73f66cd3eb18 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:54.560616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3eae5-3289-464c-95ff-73f66cd3eb18 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:54.562919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012f294-6931-419e-a13d-d6d7a478ca8b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-06d3eae5-3289-464c-95ff-73f66cd3eb18 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:37:54.576746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:54.578550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:54.580805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:54.580805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:54.580805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:37:54.580805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:37:54.580805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:37:54.580805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:37:54.580805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:37:54.580805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:37:54.580805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:37:54.580805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:54.580805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 276/551] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:37:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:54.583848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3eae5-3289-464c-95ff-73f66cd3eb18 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:54.750334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012f294-6931-419e-a13d-d6d7a478ca8b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:54.754246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3eae5-3289-464c-95ff-73f66cd3eb18 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:54.793492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e267f8-4cf0-465e-9a52-47f117ef831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:54.794805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e267f8-4cf0-465e-9a52-47f117ef831c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:54.795104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e267f8-4cf0-465e-9a52-47f117ef831c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:54.795386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e267f8-4cf0-465e-9a52-47f117ef831c None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:54.796413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e267f8-4cf0-465e-9a52-47f117ef831c None admin] RBAC: Adding query filter params (name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:54.796724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e267f8-4cf0-465e-9a52-47f117ef831c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:54.798925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e267f8-4cf0-465e-9a52-47f117ef831c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:54.810167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509e441-7ac4-45e7-bbb1-8b1067dac169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:54.811015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509e441-7ac4-45e7-bbb1-8b1067dac169 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:54.811389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 276/552] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:37:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:54.811501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509e441-7ac4-45e7-bbb1-8b1067dac169 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:54.811836 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509e441-7ac4-45e7-bbb1-8b1067dac169 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:54.812666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a64e7-2729-4313-9a4a-1a5eb0a58952 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:54.812854 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509e441-7ac4-45e7-bbb1-8b1067dac169 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:54.812955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a64e7-2729-4313-9a4a-1a5eb0a58952 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:54.813220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a64e7-2729-4313-9a4a-1a5eb0a58952 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:54.813426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509e441-7ac4-45e7-bbb1-8b1067dac169 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:54.815000 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509e441-7ac4-45e7-bbb1-8b1067dac169 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:54.818909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 277/553] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:54.819751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a64e7-2729-4313-9a4a-1a5eb0a58952 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:54.837583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 277/554] 10.222.0.81 () {56 vars in 1135 bytes} [Thu Jun 16 18:37:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:54.838934 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76844b9-7060-4196-ab83-dd9962923fbb None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:54.839224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76844b9-7060-4196-ab83-dd9962923fbb None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:54.839501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76844b9-7060-4196-ab83-dd9962923fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:54.856576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d76844b9-7060-4196-ab83-dd9962923fbb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:54.857793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d76844b9-7060-4196-ab83-dd9962923fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:54.870531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:54.881742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:54.930147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 278/555] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:54.931154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76844b9-7060-4196-ab83-dd9962923fbb None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:54.952571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a64e7-2729-4313-9a4a-1a5eb0a58952 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:55.006568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a6511-b0c1-422f-aa4c-8aaadbd3541f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:55.007542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a6511-b0c1-422f-aa4c-8aaadbd3541f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.007830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a6511-b0c1-422f-aa4c-8aaadbd3541f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.008336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a6511-b0c1-422f-aa4c-8aaadbd3541f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.022755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a6511-b0c1-422f-aa4c-8aaadbd3541f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:55.024235 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a6511-b0c1-422f-aa4c-8aaadbd3541f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-424a6511-b0c1-422f-aa4c-8aaadbd3541f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:55.048317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:55.049212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:55.050243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:55.050243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:55.050243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:55.050243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:55.052169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76844b9-7060-4196-ab83-dd9962923fbb None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:55.055071 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 278/556] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:55.056833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a6511-b0c1-422f-aa4c-8aaadbd3541f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:55.096309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556c46e-7e01-49a3-908c-d72926ef277b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:55.097530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556c46e-7e01-49a3-908c-d72926ef277b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.097936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556c46e-7e01-49a3-908c-d72926ef277b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.098216 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556c46e-7e01-49a3-908c-d72926ef277b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.117731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:55.168773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:55.172253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a6511-b0c1-422f-aa4c-8aaadbd3541f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:55.179273 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556c46e-7e01-49a3-908c-d72926ef277b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:55.180960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556c46e-7e01-49a3-908c-d72926ef277b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:55.206308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3b1be-c7d3-4e38-801b-dc707b5d0aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:55.207100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3b1be-c7d3-4e38-801b-dc707b5d0aab None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.207380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3b1be-c7d3-4e38-801b-dc707b5d0aab None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.207645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3b1be-c7d3-4e38-801b-dc707b5d0aab None admin] PATH_INFO: `/v3/roles/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.219383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3b1be-c7d3-4e38-801b-dc707b5d0aab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-bee3b1be-c7d3-4e38-801b-dc707b5d0aab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:55.227008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:55.228379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:55.229198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:55.233820 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 279/557] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:55.234535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3b1be-c7d3-4e38-801b-dc707b5d0aab None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:55.297376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 279/558] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:37:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:37:55.298627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556c46e-7e01-49a3-908c-d72926ef277b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:55.335705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3b1be-c7d3-4e38-801b-dc707b5d0aab None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:55.348719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:55.382377 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b2ed6-b602-413a-a3b9-c6b7ecf793cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:55.382997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b2ed6-b602-413a-a3b9-c6b7ecf793cb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.383355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b2ed6-b602-413a-a3b9-c6b7ecf793cb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.383702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b2ed6-b602-413a-a3b9-c6b7ecf793cb None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.384486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b2ed6-b602-413a-a3b9-c6b7ecf793cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:55.384861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b2ed6-b602-413a-a3b9-c6b7ecf793cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:55.386580 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b2ed6-b602-413a-a3b9-c6b7ecf793cb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:55.404207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 280/559] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:55.404207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b2ed6-b602-413a-a3b9-c6b7ecf793cb None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:55.418425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556c46e-7e01-49a3-908c-d72926ef277b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:55.421272 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:55.460788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d4024-7df7-4b8e-9639-192a556ac6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:55.461822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d4024-7df7-4b8e-9639-192a556ac6d4 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.462365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d4024-7df7-4b8e-9639-192a556ac6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.462818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d4024-7df7-4b8e-9639-192a556ac6d4 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.463728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d4024-7df7-4b8e-9639-192a556ac6d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:55.464755 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d4024-7df7-4b8e-9639-192a556ac6d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:55.466701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2d4024-7df7-4b8e-9639-192a556ac6d4 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:55.477277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 280/560] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:37:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:55.478407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d4024-7df7-4b8e-9639-192a556ac6d4 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:55.515966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b2ed6-b602-413a-a3b9-c6b7ecf793cb None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:55.551557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d453f-0bfb-4524-8098-f04888168b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:55.552276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d453f-0bfb-4524-8098-f04888168b3a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.552609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d453f-0bfb-4524-8098-f04888168b3a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.552887 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d453f-0bfb-4524-8098-f04888168b3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.563260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d453f-0bfb-4524-8098-f04888168b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:55.564713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d453f-0bfb-4524-8098-f04888168b3a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:55.580829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d4024-7df7-4b8e-9639-192a556ac6d4 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-827d453f-0bfb-4524-8098-f04888168b3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:55.582087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:55.582955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:55.583831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:55.583831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:55.583831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:55.583831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:55.583831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 281/561] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:55.584305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d453f-0bfb-4524-8098-f04888168b3a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:55.597210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:55.624110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb4483-3e71-4a13-bb74-4e150c20dde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:55.625449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb4483-3e71-4a13-bb74-4e150c20dde1 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.625736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb4483-3e71-4a13-bb74-4e150c20dde1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.626007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb4483-3e71-4a13-bb74-4e150c20dde1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.634882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb4483-3e71-4a13-bb74-4e150c20dde1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:55.636670 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb4483-3e71-4a13-bb74-4e150c20dde1 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-2cbb4483-3e71-4a13-bb74-4e150c20dde1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:37:55.651993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:55.653045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:55.654017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:55.654017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:55.654017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:37:55.654017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:37:55.654017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:37:55.654017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:37:55.654017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:37:55.654017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:37:55.654017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:37:55.654017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:55.654017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 281/562] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:55.654610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19fb29-c55f-4a1c-80da-e9d5d70d9285 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.655350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19fb29-c55f-4a1c-80da-e9d5d70d9285 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.655350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19fb29-c55f-4a1c-80da-e9d5d70d9285 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.656453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 282/563] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:55.657041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19fb29-c55f-4a1c-80da-e9d5d70d9285 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:55.664627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:55.699210 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d453f-0bfb-4524-8098-f04888168b3a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:55.735299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856f045-4a70-4700-9206-c1ef34ce0b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:55.736096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856f045-4a70-4700-9206-c1ef34ce0b3a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.736382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856f045-4a70-4700-9206-c1ef34ce0b3a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.736661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856f045-4a70-4700-9206-c1ef34ce0b3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.748951 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856f045-4a70-4700-9206-c1ef34ce0b3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:55.750539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856f045-4a70-4700-9206-c1ef34ce0b3a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-a856f045-4a70-4700-9206-c1ef34ce0b3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:55.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:55.768982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:55.770383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:55.770383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:55.770383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:55.770383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:55.773382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 282/564] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:55.774280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856f045-4a70-4700-9206-c1ef34ce0b3a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:55.774453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19fb29-c55f-4a1c-80da-e9d5d70d9285 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:55.809032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e321367-27ab-4b8a-9281-d5280b1b1d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:55.809823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e321367-27ab-4b8a-9281-d5280b1b1d5e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.810185 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e321367-27ab-4b8a-9281-d5280b1b1d5e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.810560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e321367-27ab-4b8a-9281-d5280b1b1d5e None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.811370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e321367-27ab-4b8a-9281-d5280b1b1d5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:55.811790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e321367-27ab-4b8a-9281-d5280b1b1d5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:55.813264 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e321367-27ab-4b8a-9281-d5280b1b1d5e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:55.825564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 283/565] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:55.826899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be22ed-ca63-4138-bcd1-c930361c1914 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.827310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be22ed-ca63-4138-bcd1-c930361c1914 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.827688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be22ed-ca63-4138-bcd1-c930361c1914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.844393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-a9be22ed-ca63-4138-bcd1-c930361c1914 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:55.845845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a9be22ed-ca63-4138-bcd1-c930361c1914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:55.871103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:55.900388 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856f045-4a70-4700-9206-c1ef34ce0b3a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:55.919323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 284/566] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:55.923090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be22ed-ca63-4138-bcd1-c930361c1914 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:55.924543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:55.936350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616de75-8dcb-4439-9299-3b3f5538e730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:55.937026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616de75-8dcb-4439-9299-3b3f5538e730 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:55.937455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616de75-8dcb-4439-9299-3b3f5538e730 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:55.937731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616de75-8dcb-4439-9299-3b3f5538e730 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:55.938674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616de75-8dcb-4439-9299-3b3f5538e730 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:55.938987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616de75-8dcb-4439-9299-3b3f5538e730 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:55.940432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616de75-8dcb-4439-9299-3b3f5538e730 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:55.962743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 283/567] 10.222.0.81 () {56 vars in 1137 bytes} [Thu Jun 16 18:37:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:55.962743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616de75-8dcb-4439-9299-3b3f5538e730 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.020506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be22ed-ca63-4138-bcd1-c930361c1914 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:56.053196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616de75-8dcb-4439-9299-3b3f5538e730 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:56.053702 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac573ac5-2941-415b-92ea-99053d94c6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:56.054360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573ac5-2941-415b-92ea-99053d94c6ba None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.054646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573ac5-2941-415b-92ea-99053d94c6ba None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.054917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac573ac5-2941-415b-92ea-99053d94c6ba None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.055624 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac573ac5-2941-415b-92ea-99053d94c6ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:56.055952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac573ac5-2941-415b-92ea-99053d94c6ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:56.057357 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac573ac5-2941-415b-92ea-99053d94c6ba None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:56.068330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 285/568] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:56.069517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49b4bd-0c8b-4649-ad61-93d67ece684f None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.069806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49b4bd-0c8b-4649-ad61-93d67ece684f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.070083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd49b4bd-0c8b-4649-ad61-93d67ece684f None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.073771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 286/569] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:56.073771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49b4bd-0c8b-4649-ad61-93d67ece684f None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.088156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6e532-7928-4dbd-894e-4eafb4a2a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:56.089526 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6e532-7928-4dbd-894e-4eafb4a2a14f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.089824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6e532-7928-4dbd-894e-4eafb4a2a14f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.090101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6e532-7928-4dbd-894e-4eafb4a2a14f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.093090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:56.096886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6e532-7928-4dbd-894e-4eafb4a2a14f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:56.098370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6e532-7928-4dbd-894e-4eafb4a2a14f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-8ce6e532-7928-4dbd-894e-4eafb4a2a14f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:37:56.111805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:56.112915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:56.119095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:37:56.119095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:37:56.119095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:37:56.119095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:37:56.119095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:37:56.119095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:37:56.119095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:56.119095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:56.119095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 284/570] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:56.119095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6e532-7928-4dbd-894e-4eafb4a2a14f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.158364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:56.176492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd49b4bd-0c8b-4649-ad61-93d67ece684f None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:56.211726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4fc78-0032-4a62-aad5-a321d3f15920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:56.212761 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4fc78-0032-4a62-aad5-a321d3f15920 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.213034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4fc78-0032-4a62-aad5-a321d3f15920 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.213333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4fc78-0032-4a62-aad5-a321d3f15920 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.215377 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6e532-7928-4dbd-894e-4eafb4a2a14f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:56.223132 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4fc78-0032-4a62-aad5-a321d3f15920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-f1c4fc78-0032-4a62-aad5-a321d3f15920 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:56.230697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:56.231763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 287/571] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:56.232957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4fc78-0032-4a62-aad5-a321d3f15920 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.250502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3b380-9dcd-4c14-9b68-09fb3d6ee45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:56.251292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3b380-9dcd-4c14-9b68-09fb3d6ee45f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.251577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3b380-9dcd-4c14-9b68-09fb3d6ee45f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.251851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3b380-9dcd-4c14-9b68-09fb3d6ee45f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.262791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3b380-9dcd-4c14-9b68-09fb3d6ee45f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:56.264227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3b380-9dcd-4c14-9b68-09fb3d6ee45f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-e7b3b380-9dcd-4c14-9b68-09fb3d6ee45f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:56.280617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:56.281691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:56.282623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:56.282623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:56.282623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:56.282623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:56.282623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 285/572] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:56.282943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170a70c-8cf0-4880-bd35-6d9c3beebf66 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.283228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170a70c-8cf0-4880-bd35-6d9c3beebf66 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.283502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170a70c-8cf0-4880-bd35-6d9c3beebf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.299151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4170a70c-8cf0-4880-bd35-6d9c3beebf66 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:56.300315 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4170a70c-8cf0-4880-bd35-6d9c3beebf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:56.317290 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:56.338490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4fc78-0032-4a62-aad5-a321d3f15920 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:56.369265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 286/573] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:56.370760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170a70c-8cf0-4880-bd35-6d9c3beebf66 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.372542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285bd57-6485-4dc6-8d04-3f6936d86504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:56.373366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285bd57-6485-4dc6-8d04-3f6936d86504 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.373649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285bd57-6485-4dc6-8d04-3f6936d86504 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.373926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285bd57-6485-4dc6-8d04-3f6936d86504 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.384578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285bd57-6485-4dc6-8d04-3f6936d86504 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:56.386805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285bd57-6485-4dc6-8d04-3f6936d86504 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:56.395734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-1285bd57-6485-4dc6-8d04-3f6936d86504 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:37:56.400489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:56.410468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 288/574] 10.222.0.81 () {56 vars in 1093 bytes} [Thu Jun 16 18:37:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:56.411646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285bd57-6485-4dc6-8d04-3f6936d86504 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.475915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4170a70c-8cf0-4880-bd35-6d9c3beebf66 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:56.500328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285bd57-6485-4dc6-8d04-3f6936d86504 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:56.508511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be3287-7974-4670-8872-826371dbd2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:56.509161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be3287-7974-4670-8872-826371dbd2ee None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.509472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be3287-7974-4670-8872-826371dbd2ee None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.509746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be3287-7974-4670-8872-826371dbd2ee None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.510520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be3287-7974-4670-8872-826371dbd2ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:56.510878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be3287-7974-4670-8872-826371dbd2ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:56.512806 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be3287-7974-4670-8872-826371dbd2ee None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:56.523767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 287/575] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:37:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:56.524532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be3287-7974-4670-8872-826371dbd2ee None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.536674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c91566-89ba-4e92-8bd7-2a184d50a995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:56.537357 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c91566-89ba-4e92-8bd7-2a184d50a995 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.537642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c91566-89ba-4e92-8bd7-2a184d50a995 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.537915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c91566-89ba-4e92-8bd7-2a184d50a995 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.538643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c91566-89ba-4e92-8bd7-2a184d50a995 None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:56.538944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c91566-89ba-4e92-8bd7-2a184d50a995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:56.540305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c91566-89ba-4e92-8bd7-2a184d50a995 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:56.545722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:56.553337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 289/576] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:37:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:56.553337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c91566-89ba-4e92-8bd7-2a184d50a995 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.627044 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:56.646502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be3287-7974-4670-8872-826371dbd2ee None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:56.657329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c91566-89ba-4e92-8bd7-2a184d50a995 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:56.682133 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ace7f-751f-4492-b13e-22f1d2b77973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:56.683339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ace7f-751f-4492-b13e-22f1d2b77973 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.683630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ace7f-751f-4492-b13e-22f1d2b77973 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.684227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ace7f-751f-4492-b13e-22f1d2b77973 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.684603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ace7f-751f-4492-b13e-22f1d2b77973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:56.684908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ace7f-751f-4492-b13e-22f1d2b77973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:56.686328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ace7f-751f-4492-b13e-22f1d2b77973 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:56.697924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 288/577] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:56.699384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3c71c-bc45-4895-a464-be19881091e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:56.700240 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3c71c-bc45-4895-a464-be19881091e2 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.700524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3c71c-bc45-4895-a464-be19881091e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.700799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3c71c-bc45-4895-a464-be19881091e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.702127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ace7f-751f-4492-b13e-22f1d2b77973 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.715231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3c71c-bc45-4895-a464-be19881091e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-e3c3c71c-bc45-4895-a464-be19881091e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:56.726628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:56.727630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:56.728455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 290/578] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:56.729160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3c71c-bc45-4895-a464-be19881091e2 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.794485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:56.808627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ace7f-751f-4492-b13e-22f1d2b77973 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:56.839496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3c71c-bc45-4895-a464-be19881091e2 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:56.843110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62547651-341f-4f90-9513-e0df13ca593b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:56.843871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547651-341f-4f90-9513-e0df13ca593b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.844159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547651-341f-4f90-9513-e0df13ca593b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.844433 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547651-341f-4f90-9513-e0df13ca593b None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.845279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62547651-341f-4f90-9513-e0df13ca593b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:56.845595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62547651-341f-4f90-9513-e0df13ca593b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:56.847020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62547651-341f-4f90-9513-e0df13ca593b None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:56.862481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:56.865840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 289/579] 10.222.0.81 () {56 vars in 1133 bytes} [Thu Jun 16 18:37:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:56.866622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62547651-341f-4f90-9513-e0df13ca593b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.882240 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6b742-e46f-4bf1-a805-601cf367bf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:56.882894 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6b742-e46f-4bf1-a805-601cf367bf2b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:56.883180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6b742-e46f-4bf1-a805-601cf367bf2b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:56.883457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c6b742-e46f-4bf1-a805-601cf367bf2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:56.884358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6b742-e46f-4bf1-a805-601cf367bf2b None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=b90f65f1f04b416286142c081c92a02d) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:56.884748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6b742-e46f-4bf1-a805-601cf367bf2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:56.886171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c6b742-e46f-4bf1-a805-601cf367bf2b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:56.891568 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 291/580] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:37:56 2022] GET /identity/v3/role_assignments?user.id=b90f65f1f04b416286142c081c92a02d&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:56.892346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6b742-e46f-4bf1-a805-601cf367bf2b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:56.975479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62547651-341f-4f90-9513-e0df13ca593b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.007528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c6b742-e46f-4bf1-a805-601cf367bf2b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.017821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a31c9-76b1-4c12-a89d-3b08524af3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.018944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a31c9-76b1-4c12-a89d-3b08524af3a5 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.019229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a31c9-76b1-4c12-a89d-3b08524af3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.019663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a31c9-76b1-4c12-a89d-3b08524af3a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:57.031606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a31c9-76b1-4c12-a89d-3b08524af3a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:57.033137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a31c9-76b1-4c12-a89d-3b08524af3a5 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:57.033796 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:57.045565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf9557-1c95-491b-81f3-df3afbbe76e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.046365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf9557-1c95-491b-81f3-df3afbbe76e4 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.047306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf9557-1c95-491b-81f3-df3afbbe76e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.047592 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daf9557-1c95-491b-81f3-df3afbbe76e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-c36a31c9-76b1-4c12-a89d-3b08524af3a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:57.052792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:57.053770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:57.054665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:57.054665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:57.054665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:57.054665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:57.054665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 290/581] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:57.054948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a31c9-76b1-4c12-a89d-3b08524af3a5 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:57.059348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf9557-1c95-491b-81f3-df3afbbe76e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:57.060937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daf9557-1c95-491b-81f3-df3afbbe76e4 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-7daf9557-1c95-491b-81f3-df3afbbe76e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:37:57.076188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:57.077119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:57.078119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:37:57.078119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:37:57.078119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:37:57.078119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:37:57.078119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:37:57.078119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:37:57.078119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:57.078119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:57.078119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 292/582] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:57.078599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf9557-1c95-491b-81f3-df3afbbe76e4 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:57.114110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:57.169462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a31c9-76b1-4c12-a89d-3b08524af3a5 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.186100 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daf9557-1c95-491b-81f3-df3afbbe76e4 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.203168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710ac24-659d-4366-96b1-c1619d3c98cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.203837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710ac24-659d-4366-96b1-c1619d3c98cc None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.204124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710ac24-659d-4366-96b1-c1619d3c98cc None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.204408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710ac24-659d-4366-96b1-c1619d3c98cc None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:57.205215 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710ac24-659d-4366-96b1-c1619d3c98cc None admin] RBAC: Adding query filter params (name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:57.205568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710ac24-659d-4366-96b1-c1619d3c98cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:57.207007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710ac24-659d-4366-96b1-c1619d3c98cc None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:57.219820 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 291/583] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:37:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:57.219820 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710ac24-659d-4366-96b1-c1619d3c98cc None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:57.226028 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f431d6-fd95-48e8-a66f-3a3a495c173b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.226916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f431d6-fd95-48e8-a66f-3a3a495c173b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.227268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f431d6-fd95-48e8-a66f-3a3a495c173b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.227604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f431d6-fd95-48e8-a66f-3a3a495c173b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:57.238479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f431d6-fd95-48e8-a66f-3a3a495c173b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:57.239918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f431d6-fd95-48e8-a66f-3a3a495c173b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-99f431d6-fd95-48e8-a66f-3a3a495c173b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:57.257465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:57.258575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:57.259494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:57.259494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:57.259494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:57.259494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:57.259494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 293/584] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:57.259810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f431d6-fd95-48e8-a66f-3a3a495c173b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:57.299033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:57.325942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710ac24-659d-4366-96b1-c1619d3c98cc None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.341159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:57.367009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df2d2-dc20-47a3-9e6a-c3a9968a76b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.367684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df2d2-dc20-47a3-9e6a-c3a9968a76b1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.367971 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df2d2-dc20-47a3-9e6a-c3a9968a76b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.368251 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df2d2-dc20-47a3-9e6a-c3a9968a76b1 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:57.369311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0df2d2-dc20-47a3-9e6a-c3a9968a76b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:57.369563 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0df2d2-dc20-47a3-9e6a-c3a9968a76b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:57.371518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0df2d2-dc20-47a3-9e6a-c3a9968a76b1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:57.374961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f431d6-fd95-48e8-a66f-3a3a495c173b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.384128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 292/585] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:57.389061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df2d2-dc20-47a3-9e6a-c3a9968a76b1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:57.410169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a267be-ee0d-4f83-86e2-6398a272d12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.410937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a267be-ee0d-4f83-86e2-6398a272d12b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.411220 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a267be-ee0d-4f83-86e2-6398a272d12b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.411488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a267be-ee0d-4f83-86e2-6398a272d12b None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:57.412255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a267be-ee0d-4f83-86e2-6398a272d12b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:57.412560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a267be-ee0d-4f83-86e2-6398a272d12b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:57.414027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a267be-ee0d-4f83-86e2-6398a272d12b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:57.430750 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 294/586] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:37:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:57.430750 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a267be-ee0d-4f83-86e2-6398a272d12b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:57.493187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df2d2-dc20-47a3-9e6a-c3a9968a76b1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.527312 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:57.528594 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f6d61-e9b1-46f4-90e8-1ffafa6fdd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.529382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f6d61-e9b1-46f4-90e8-1ffafa6fdd4e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.529674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f6d61-e9b1-46f4-90e8-1ffafa6fdd4e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.529952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f6d61-e9b1-46f4-90e8-1ffafa6fdd4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:57.534019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a267be-ee0d-4f83-86e2-6398a272d12b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.540395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f6d61-e9b1-46f4-90e8-1ffafa6fdd4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:57.541876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f6d61-e9b1-46f4-90e8-1ffafa6fdd4e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-f10f6d61-e9b1-46f4-90e8-1ffafa6fdd4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:57.559428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:57.560373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:57.561298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:57.561298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:57.561298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:57.561298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:57.565212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 293/587] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:57.566061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f6d61-e9b1-46f4-90e8-1ffafa6fdd4e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:57.568755 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243eb56-c9a6-4e7f-9978-0368d314df4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.569452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243eb56-c9a6-4e7f-9978-0368d314df4a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.569737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243eb56-c9a6-4e7f-9978-0368d314df4a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.570008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243eb56-c9a6-4e7f-9978-0368d314df4a None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:57.570716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243eb56-c9a6-4e7f-9978-0368d314df4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:57.571020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243eb56-c9a6-4e7f-9978-0368d314df4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:57.572378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243eb56-c9a6-4e7f-9978-0368d314df4a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:57.589904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 295/588] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:57.589904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243eb56-c9a6-4e7f-9978-0368d314df4a None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:57.600669 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:57.681788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f6d61-e9b1-46f4-90e8-1ffafa6fdd4e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.693295 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243eb56-c9a6-4e7f-9978-0368d314df4a None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.721933 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43111e63-e186-4f3c-8d20-210acd050a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.722731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43111e63-e186-4f3c-8d20-210acd050a07 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.723168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43111e63-e186-4f3c-8d20-210acd050a07 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.723353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43111e63-e186-4f3c-8d20-210acd050a07 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:57.727720 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b676015-5506-4afc-904d-1d576704c67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.728597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b676015-5506-4afc-904d-1d576704c67d None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.728878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b676015-5506-4afc-904d-1d576704c67d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.729149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b676015-5506-4afc-904d-1d576704c67d None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/d17b7a7128d64b2ba548add02d3ec264/roles/9718ec052fba437f923e8b15e08d7bed` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:57.731419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43111e63-e186-4f3c-8d20-210acd050a07 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:57.733119 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43111e63-e186-4f3c-8d20-210acd050a07 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:57.744768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b676015-5506-4afc-904d-1d576704c67d None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=d17b7a7128d64b2ba548add02d3ec264, role_id=9718ec052fba437f923e8b15e08d7bed)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:57.746754 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b676015-5506-4afc-904d-1d576704c67d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-43111e63-e186-4f3c-8d20-210acd050a07 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:37:57.748252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:57.749223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 294/589] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:57.750204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43111e63-e186-4f3c-8d20-210acd050a07 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:57.829272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 296/590] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:37:57 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/d17b7a7128d64b2ba548add02d3ec264/roles/9718ec052fba437f923e8b15e08d7bed => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:57.829272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b676015-5506-4afc-904d-1d576704c67d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:57.836847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:57.850501 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:57.870856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43111e63-e186-4f3c-8d20-210acd050a07 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.927166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25b981-dbcf-43b4-8434-fdd2b3dca091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.927858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25b981-dbcf-43b4-8434-fdd2b3dca091 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.928147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25b981-dbcf-43b4-8434-fdd2b3dca091 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.928610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25b981-dbcf-43b4-8434-fdd2b3dca091 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:57.929730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25b981-dbcf-43b4-8434-fdd2b3dca091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:57.930054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25b981-dbcf-43b4-8434-fdd2b3dca091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:57.931545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25b981-dbcf-43b4-8434-fdd2b3dca091 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:57.947401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b676015-5506-4afc-904d-1d576704c67d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:57.951989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 295/591] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:57.952960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25b981-dbcf-43b4-8434-fdd2b3dca091 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:57.990766 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4019e-2260-490f-b8e8-a464fe9bdd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:57.991744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4019e-2260-490f-b8e8-a464fe9bdd05 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:57.992122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4019e-2260-490f-b8e8-a464fe9bdd05 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:57.992488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4019e-2260-490f-b8e8-a464fe9bdd05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:58.002071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4019e-2260-490f-b8e8-a464fe9bdd05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:58.002805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4019e-2260-490f-b8e8-a464fe9bdd05 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-acf4019e-2260-490f-b8e8-a464fe9bdd05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:37:58.020764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:58.022792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:58.023888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:37:58.023888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:37:58.023888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:37:58.023888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:37:58.023888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:37:58.023888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:37:58.023888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:58.023888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:58.026141 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 297/592] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:58.027150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4019e-2260-490f-b8e8-a464fe9bdd05 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:58.088128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25b981-dbcf-43b4-8434-fdd2b3dca091 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:58.111698 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:58.133637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445b86f-4804-4a3b-9f29-919118b22c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:58.134323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b86f-4804-4a3b-9f29-919118b22c9e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:58.134608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b86f-4804-4a3b-9f29-919118b22c9e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:58.134884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445b86f-4804-4a3b-9f29-919118b22c9e None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:58.135720 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445b86f-4804-4a3b-9f29-919118b22c9e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:58.136068 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445b86f-4804-4a3b-9f29-919118b22c9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:58.137593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445b86f-4804-4a3b-9f29-919118b22c9e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:58.155476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 296/593] 10.222.0.81 () {56 vars in 1139 bytes} [Thu Jun 16 18:37:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:58.156305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445b86f-4804-4a3b-9f29-919118b22c9e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:58.157941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4019e-2260-490f-b8e8-a464fe9bdd05 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:58.160365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:58.196419 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159181d-b009-4f18-9e10-d7148682f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:58.197559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159181d-b009-4f18-9e10-d7148682f885 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:58.198002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159181d-b009-4f18-9e10-d7148682f885 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:58.198453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159181d-b009-4f18-9e10-d7148682f885 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:58.207081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159181d-b009-4f18-9e10-d7148682f885 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:58.208723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159181d-b009-4f18-9e10-d7148682f885 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-5159181d-b009-4f18-9e10-d7148682f885 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:37:58.223524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:58.224668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:58.225780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:58.225780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:58.225780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:37:58.225780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:37:58.225780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:37:58.225780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:37:58.225780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:37:58.225780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:37:58.225780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:37:58.225780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:58.227943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 298/594] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:37:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:58.228940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159181d-b009-4f18-9e10-d7148682f885 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:58.264983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445b86f-4804-4a3b-9f29-919118b22c9e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:58.299195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2066b8-e05d-424f-a026-b8147ca7a7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:58.300110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2066b8-e05d-424f-a026-b8147ca7a7d9 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:58.300355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2066b8-e05d-424f-a026-b8147ca7a7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:58.300695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2066b8-e05d-424f-a026-b8147ca7a7d9 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:58.301636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2066b8-e05d-424f-a026-b8147ca7a7d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:58.301987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2066b8-e05d-424f-a026-b8147ca7a7d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:58.303543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2066b8-e05d-424f-a026-b8147ca7a7d9 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:58.321132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 297/595] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:58.321132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2066b8-e05d-424f-a026-b8147ca7a7d9 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:58.351989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159181d-b009-4f18-9e10-d7148682f885 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:58.380347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:58.390932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff0fd0-a00d-42a6-a1ff-4a8303466635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:58.391980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff0fd0-a00d-42a6-a1ff-4a8303466635 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:58.392272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff0fd0-a00d-42a6-a1ff-4a8303466635 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:58.392547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff0fd0-a00d-42a6-a1ff-4a8303466635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:58.396823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:58.407639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff0fd0-a00d-42a6-a1ff-4a8303466635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:58.409107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff0fd0-a00d-42a6-a1ff-4a8303466635 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-62ff0fd0-a00d-42a6-a1ff-4a8303466635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:58.431452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:58.432579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:58.433776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:58.433776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:58.433776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:58.433776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:58.433776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 299/596] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:58.433776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff0fd0-a00d-42a6-a1ff-4a8303466635 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:58.441708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2066b8-e05d-424f-a026-b8147ca7a7d9 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:58.484731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc5a9-a7b2-435c-9765-e6cdbf73a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:58.485837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc5a9-a7b2-435c-9765-e6cdbf73a9cf None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:58.486129 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc5a9-a7b2-435c-9765-e6cdbf73a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:58.486404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc5a9-a7b2-435c-9765-e6cdbf73a9cf None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:58.487287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cc5a9-a7b2-435c-9765-e6cdbf73a9cf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:58.488046 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cc5a9-a7b2-435c-9765-e6cdbf73a9cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:58.489957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cc5a9-a7b2-435c-9765-e6cdbf73a9cf None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:58.517832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 298/597] 10.222.0.81 () {56 vars in 1137 bytes} [Thu Jun 16 18:37:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:58.520242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc5a9-a7b2-435c-9765-e6cdbf73a9cf None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:58.556751 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff0fd0-a00d-42a6-a1ff-4a8303466635 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:58.592984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f9d2b-f7a9-42b2-bfe5-2d1163154109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:58.593953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f9d2b-f7a9-42b2-bfe5-2d1163154109 None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:58.594244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f9d2b-f7a9-42b2-bfe5-2d1163154109 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:58.594498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f9d2b-f7a9-42b2-bfe5-2d1163154109 None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/251ed026ce8046e98188066bf0bc40b7/roles/9718ec052fba437f923e8b15e08d7bed` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:58.610576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f9d2b-f7a9-42b2-bfe5-2d1163154109 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=251ed026ce8046e98188066bf0bc40b7, role_id=9718ec052fba437f923e8b15e08d7bed)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:58.612047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f9d2b-f7a9-42b2-bfe5-2d1163154109 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:58.646928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc5a9-a7b2-435c-9765-e6cdbf73a9cf None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:58.672388 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:58.693275 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 300/598] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:37:58 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/251ed026ce8046e98188066bf0bc40b7/roles/9718ec052fba437f923e8b15e08d7bed => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:58.693275 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f9d2b-f7a9-42b2-bfe5-2d1163154109 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:58.697953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c9b7d-9417-47d1-8370-f07acb759b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:58.698698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c9b7d-9417-47d1-8370-f07acb759b47 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:58.699087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c9b7d-9417-47d1-8370-f07acb759b47 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:58.700375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c9b7d-9417-47d1-8370-f07acb759b47 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:58.700375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c9b7d-9417-47d1-8370-f07acb759b47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:58.700655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c9b7d-9417-47d1-8370-f07acb759b47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:58.702164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c9b7d-9417-47d1-8370-f07acb759b47 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:58.734006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 299/599] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:58.738463 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c9b7d-9417-47d1-8370-f07acb759b47 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:58.761559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:58.903577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f9d2b-f7a9-42b2-bfe5-2d1163154109 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:58.933007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c9b7d-9417-47d1-8370-f07acb759b47 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:58.947698 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfee5c-8001-421c-baa0-a8e833d4c0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:58.949090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfee5c-8001-421c-baa0-a8e833d4c0bd None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:58.949418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfee5c-8001-421c-baa0-a8e833d4c0bd None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:58.949693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfee5c-8001-421c-baa0-a8e833d4c0bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:58.961252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfee5c-8001-421c-baa0-a8e833d4c0bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:58.962728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfee5c-8001-421c-baa0-a8e833d4c0bd None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:58.974411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c1efa-e642-4312-9ced-4afa2d9623a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:58.975562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c1efa-e642-4312-9ced-4afa2d9623a3 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:58.976051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c1efa-e642-4312-9ced-4afa2d9623a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:58.976527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c1efa-e642-4312-9ced-4afa2d9623a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-6edfee5c-8001-421c-baa0-a8e833d4c0bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:37:58.984136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:58.985049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:58.986285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:37:58.986285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:37:58.986285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:37:58.986285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:58.986285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 301/600] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:58.986730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c1efa-e642-4312-9ced-4afa2d9623a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:58.988353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c1efa-e642-4312-9ced-4afa2d9623a3 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:58.991246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfee5c-8001-421c-baa0-a8e833d4c0bd None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-660c1efa-e642-4312-9ced-4afa2d9623a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:37:59.004620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:59.005871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:59.007137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:37:59.007137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:37:59.007137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:37:59.007137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:37:59.007137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:37:59.007137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:37:59.007137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:59.007137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:37:59.007137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 300/601] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:59.012215 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c1efa-e642-4312-9ced-4afa2d9623a3 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:59.075718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:59.099359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:59.128508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfee5c-8001-421c-baa0-a8e833d4c0bd None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:59.144875 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c1efa-e642-4312-9ced-4afa2d9623a3 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:59.172965 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d9220-7b59-4151-844e-057db59bf5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:59.174131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d9220-7b59-4151-844e-057db59bf5a2 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:59.174497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d9220-7b59-4151-844e-057db59bf5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:59.174781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d9220-7b59-4151-844e-057db59bf5a2 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:59.175556 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d9220-7b59-4151-844e-057db59bf5a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:59.176028 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d9220-7b59-4151-844e-057db59bf5a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:59.177780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d9220-7b59-4151-844e-057db59bf5a2 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:59.191737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb8da9-9f28-42df-be0c-9250fde4ba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:59.192417 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 302/602] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:59.192988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb8da9-9f28-42df-be0c-9250fde4ba9a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:59.193503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb8da9-9f28-42df-be0c-9250fde4ba9a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:59.193979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb8da9-9f28-42df-be0c-9250fde4ba9a None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:59.194970 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb8da9-9f28-42df-be0c-9250fde4ba9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:59.195480 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb8da9-9f28-42df-be0c-9250fde4ba9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:59.197131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb8da9-9f28-42df-be0c-9250fde4ba9a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:59.197975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020768ab-7dad-42e3-b124-67d495331c14 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:59.198276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020768ab-7dad-42e3-b124-67d495331c14 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:59.198564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020768ab-7dad-42e3-b124-67d495331c14 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:59.200434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 303/603] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:37:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:37:59.202822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020768ab-7dad-42e3-b124-67d495331c14 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:59.214903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 301/604] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:37:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:37:59.216952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b946f7b-e627-44fa-8f55-5d7948e74037 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:59.217930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b946f7b-e627-44fa-8f55-5d7948e74037 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:59.218417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b946f7b-e627-44fa-8f55-5d7948e74037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:59.241053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4b946f7b-e627-44fa-8f55-5d7948e74037 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:37:59.242621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4b946f7b-e627-44fa-8f55-5d7948e74037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:37:59.323895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 302/605] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:37:59.325068 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b946f7b-e627-44fa-8f55-5d7948e74037 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:59.339035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020768ab-7dad-42e3-b124-67d495331c14 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:59.374487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:59.377879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3b394-feb8-4b3c-92b2-568db6097206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:59.378685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3b394-feb8-4b3c-92b2-568db6097206 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:59.378971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3b394-feb8-4b3c-92b2-568db6097206 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:59.379247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde3b394-feb8-4b3c-92b2-568db6097206 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:59.386682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3b394-feb8-4b3c-92b2-568db6097206 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:59.388133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde3b394-feb8-4b3c-92b2-568db6097206 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:59.390900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-fde3b394-feb8-4b3c-92b2-568db6097206 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:37:59.403604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:59.404763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:59.405920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:37:59.405920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:37:59.405920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:37:59.405920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:37:59.405920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:37:59.405920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:37:59.405920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:37:59.405920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:59.406624 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 304/606] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:37:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:37:59.409261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3b394-feb8-4b3c-92b2-568db6097206 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:59.446218 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b946f7b-e627-44fa-8f55-5d7948e74037 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:59.481575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cb61c-4044-4a6a-9360-c42a7bb98cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:59.482464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cb61c-4044-4a6a-9360-c42a7bb98cc0 None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:59.482752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cb61c-4044-4a6a-9360-c42a7bb98cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:59.483030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cb61c-4044-4a6a-9360-c42a7bb98cc0 None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/f4e05d75cd1142e18d7518b78edd88fa/roles/9718ec052fba437f923e8b15e08d7bed` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:59.499957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cb61c-4044-4a6a-9360-c42a7bb98cc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=f4e05d75cd1142e18d7518b78edd88fa, role_id=9718ec052fba437f923e8b15e08d7bed)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:59.501461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cb61c-4044-4a6a-9360-c42a7bb98cc0 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:59.521034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde3b394-feb8-4b3c-92b2-568db6097206 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:59.557799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518174f0-b12d-414e-8005-499fc3c638f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:59.558627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518174f0-b12d-414e-8005-499fc3c638f9 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:59.558913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518174f0-b12d-414e-8005-499fc3c638f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:59.559192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518174f0-b12d-414e-8005-499fc3c638f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:59.569942 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518174f0-b12d-414e-8005-499fc3c638f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-518174f0-b12d-414e-8005-499fc3c638f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:37:59.578567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:37:59.579775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:37:59.580641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 305/607] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:37:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:37:59.585770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518174f0-b12d-414e-8005-499fc3c638f9 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:59.586845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 303/608] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:37:59 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/f4e05d75cd1142e18d7518b78edd88fa/roles/9718ec052fba437f923e8b15e08d7bed => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:37:59.588160 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cb61c-4044-4a6a-9360-c42a7bb98cc0 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:59.696758 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:59.729486 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:37:59.739757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cb61c-4044-4a6a-9360-c42a7bb98cc0 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:59.785057 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042474d-e4a9-4f94-8eee-7aec0f0efe14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:59.785689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042474d-e4a9-4f94-8eee-7aec0f0efe14 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:59.785973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042474d-e4a9-4f94-8eee-7aec0f0efe14 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:59.786249 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b042474d-e4a9-4f94-8eee-7aec0f0efe14 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:59.786998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042474d-e4a9-4f94-8eee-7aec0f0efe14 None admin] RBAC: Adding query filter params (name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:59.787520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042474d-e4a9-4f94-8eee-7aec0f0efe14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:59.788904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b042474d-e4a9-4f94-8eee-7aec0f0efe14 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:59.794528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518174f0-b12d-414e-8005-499fc3c638f9 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:59.801467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 304/609] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:37:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:59.804116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042474d-e4a9-4f94-8eee-7aec0f0efe14 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:59.840457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f01b8-18de-41ce-8321-f8221512c535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:59.841127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f01b8-18de-41ce-8321-f8221512c535 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:59.841438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f01b8-18de-41ce-8321-f8221512c535 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:59.841717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f01b8-18de-41ce-8321-f8221512c535 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:59.842496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f01b8-18de-41ce-8321-f8221512c535 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:37:59.842802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f01b8-18de-41ce-8321-f8221512c535 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:59.844249 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f01b8-18de-41ce-8321-f8221512c535 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:59.870145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 306/610] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:37:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:37:59.875884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f01b8-18de-41ce-8321-f8221512c535 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:37:59.931842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b042474d-e4a9-4f94-8eee-7aec0f0efe14 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:37:59.968054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f08e7a-afed-468c-8cbb-ec91279d3aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:37:59.968858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f08e7a-afed-468c-8cbb-ec91279d3aa3 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:37:59.969143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f08e7a-afed-468c-8cbb-ec91279d3aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:37:59.969448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f08e7a-afed-468c-8cbb-ec91279d3aa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:37:59.982452 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f08e7a-afed-468c-8cbb-ec91279d3aa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:37:59.984354 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f08e7a-afed-468c-8cbb-ec91279d3aa3 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:37:59.987477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-39f08e7a-afed-468c-8cbb-ec91279d3aa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:00.005616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:00.006631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:00.007495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:00.007495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:00.007495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:00.007495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:00.007495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 305/611] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:00.015146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f08e7a-afed-468c-8cbb-ec91279d3aa3 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:00.127571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f01b8-18de-41ce-8321-f8221512c535 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:00.171080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:00.197947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672c4f6-2f01-45af-a357-9a0cb587098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:00.199419 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672c4f6-2f01-45af-a357-9a0cb587098e None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:00.199708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672c4f6-2f01-45af-a357-9a0cb587098e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:00.199984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672c4f6-2f01-45af-a357-9a0cb587098e None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/9c82335b7ec74fb3a1508a2fc76335ab/roles/9718ec052fba437f923e8b15e08d7bed` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:00.207006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f08e7a-afed-468c-8cbb-ec91279d3aa3 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:00.215802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672c4f6-2f01-45af-a357-9a0cb587098e None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=9c82335b7ec74fb3a1508a2fc76335ab, role_id=9718ec052fba437f923e8b15e08d7bed)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:00.217307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672c4f6-2f01-45af-a357-9a0cb587098e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:00.244162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95677a04-3187-4b1c-a2cf-bb9364212d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:00.245199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95677a04-3187-4b1c-a2cf-bb9364212d1b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:00.246345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95677a04-3187-4b1c-a2cf-bb9364212d1b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:00.246919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95677a04-3187-4b1c-a2cf-bb9364212d1b None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:00.247818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95677a04-3187-4b1c-a2cf-bb9364212d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:00.248312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95677a04-3187-4b1c-a2cf-bb9364212d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:00.250109 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95677a04-3187-4b1c-a2cf-bb9364212d1b None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:00.264035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 306/612] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:00.265707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6743-460d-4787-899d-d5651bf32cf7 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:00.266167 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6743-460d-4787-899d-d5651bf32cf7 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:00.266652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6743-460d-4787-899d-d5651bf32cf7 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:00.272086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 307/613] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:00.272086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b6743-460d-4787-899d-d5651bf32cf7 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:00.285671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 307/614] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:37:59 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/9c82335b7ec74fb3a1508a2fc76335ab/roles/9718ec052fba437f923e8b15e08d7bed => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:38:00.286946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0f16b-aefc-4d55-b5b0-5010da5d738d None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:00.287236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0f16b-aefc-4d55-b5b0-5010da5d738d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:00.287540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0f16b-aefc-4d55-b5b0-5010da5d738d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:00.327154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-bbd0f16b-aefc-4d55-b5b0-5010da5d738d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:00.328825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-bbd0f16b-aefc-4d55-b5b0-5010da5d738d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:00.372554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:00.478556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b6743-460d-4787-899d-d5651bf32cf7 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:00.483439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 308/615] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:00.522562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1860bab-2c0d-47d6-a6b5-52a949be8fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:00.524567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0f16b-aefc-4d55-b5b0-5010da5d738d None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:00.524755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860bab-2c0d-47d6-a6b5-52a949be8fa4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:00.525054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860bab-2c0d-47d6-a6b5-52a949be8fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:00.525370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1860bab-2c0d-47d6-a6b5-52a949be8fa4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:00.533149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1860bab-2c0d-47d6-a6b5-52a949be8fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:00.534636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1860bab-2c0d-47d6-a6b5-52a949be8fa4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-f1860bab-2c0d-47d6-a6b5-52a949be8fa4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:00.552494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:00.553519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:00.555619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:00.555619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:00.555619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:00.555619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:00.555619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:00.555619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:00.555619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:00.555619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:00.555619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:38:00.555619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:00.555619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 308/616] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:00.560151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1860bab-2c0d-47d6-a6b5-52a949be8fa4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:00.639780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:00.643692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0f16b-aefc-4d55-b5b0-5010da5d738d None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:00.663807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1860bab-2c0d-47d6-a6b5-52a949be8fa4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:00.681600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:00.683740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60928-31e0-4f93-a88c-d0d03ababe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:00.684857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60928-31e0-4f93-a88c-d0d03ababe3c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:00.685398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60928-31e0-4f93-a88c-d0d03ababe3c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:00.685915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b60928-31e0-4f93-a88c-d0d03ababe3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:00.697864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b60928-31e0-4f93-a88c-d0d03ababe3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:00.699653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b65a7d-3ade-4e29-92b3-6f10c249f72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:00.701983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b65a7d-3ade-4e29-92b3-6f10c249f72a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:00.701983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b65a7d-3ade-4e29-92b3-6f10c249f72a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:00.701983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b65a7d-3ade-4e29-92b3-6f10c249f72a None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:00.701983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b65a7d-3ade-4e29-92b3-6f10c249f72a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:00.701983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b65a7d-3ade-4e29-92b3-6f10c249f72a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:00.703527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b65a7d-3ade-4e29-92b3-6f10c249f72a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-f2b60928-31e0-4f93-a88c-d0d03ababe3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:00.708131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:00.709093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:00.709973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:00.712868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 309/617] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:38:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:00.721655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60928-31e0-4f93-a88c-d0d03ababe3c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:00.729665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 309/618] 10.222.0.81 () {56 vars in 1135 bytes} [Thu Jun 16 18:38:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:00.733150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b65a7d-3ade-4e29-92b3-6f10c249f72a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:00.831868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b60928-31e0-4f93-a88c-d0d03ababe3c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:00.870677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabc5f1-c6ba-4447-8717-0a5f4ebc43d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:00.871817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc5f1-c6ba-4447-8717-0a5f4ebc43d2 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:00.872331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc5f1-c6ba-4447-8717-0a5f4ebc43d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:00.872805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc5f1-c6ba-4447-8717-0a5f4ebc43d2 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:00.873788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabc5f1-c6ba-4447-8717-0a5f4ebc43d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:00.874300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabc5f1-c6ba-4447-8717-0a5f4ebc43d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:00.875895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfabc5f1-c6ba-4447-8717-0a5f4ebc43d2 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:00.886013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 310/619] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:38:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:00.891469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabc5f1-c6ba-4447-8717-0a5f4ebc43d2 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:00.943477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:00.970630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b65a7d-3ade-4e29-92b3-6f10c249f72a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:01.011816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082ed6a-6003-4b6b-88f8-7ab2ae5fa400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:01.012994 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082ed6a-6003-4b6b-88f8-7ab2ae5fa400 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.013531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082ed6a-6003-4b6b-88f8-7ab2ae5fa400 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.014030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082ed6a-6003-4b6b-88f8-7ab2ae5fa400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.025886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082ed6a-6003-4b6b-88f8-7ab2ae5fa400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:01.028186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082ed6a-6003-4b6b-88f8-7ab2ae5fa400 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-2082ed6a-6003-4b6b-88f8-7ab2ae5fa400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:01.052402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:01.053405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:01.055517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:01.055517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:01.055517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:01.055517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:01.055517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 310/620] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:01.059415 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082ed6a-6003-4b6b-88f8-7ab2ae5fa400 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:01.112855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:01.114645 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabc5f1-c6ba-4447-8717-0a5f4ebc43d2 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:01.180635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763fec0-8496-49c7-959f-6b15137fcaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:01.181469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763fec0-8496-49c7-959f-6b15137fcaf1 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.181759 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763fec0-8496-49c7-959f-6b15137fcaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.182036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763fec0-8496-49c7-959f-6b15137fcaf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.204588 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763fec0-8496-49c7-959f-6b15137fcaf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:01.206109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9763fec0-8496-49c7-959f-6b15137fcaf1 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-9763fec0-8496-49c7-959f-6b15137fcaf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:01.236196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:01.237162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:01.238108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:01.238108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:01.238108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:01.238108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:01.241020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 311/621] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:01.252749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763fec0-8496-49c7-959f-6b15137fcaf1 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:01.262874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082ed6a-6003-4b6b-88f8-7ab2ae5fa400 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:01.312184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0127d-1d66-4051-8730-9f5299215e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:01.313284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0127d-1d66-4051-8730-9f5299215e01 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.313854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0127d-1d66-4051-8730-9f5299215e01 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.314138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0127d-1d66-4051-8730-9f5299215e01 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.314861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0127d-1d66-4051-8730-9f5299215e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:01.315423 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0127d-1d66-4051-8730-9f5299215e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:01.317561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0127d-1d66-4051-8730-9f5299215e01 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:01.334147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 311/622] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:01.335476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f1572-3364-47c2-97c0-78bb8641bcbe None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.340949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f1572-3364-47c2-97c0-78bb8641bcbe None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.341280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f1572-3364-47c2-97c0-78bb8641bcbe None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.343325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 312/623] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:01.344222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f1572-3364-47c2-97c0-78bb8641bcbe None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:01.356372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:01.385734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763fec0-8496-49c7-959f-6b15137fcaf1 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:01.421121 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7989436-d773-4dca-b2bc-74467b891486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:01.421859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7989436-d773-4dca-b2bc-74467b891486 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.422152 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7989436-d773-4dca-b2bc-74467b891486 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.422426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7989436-d773-4dca-b2bc-74467b891486 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.423185 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7989436-d773-4dca-b2bc-74467b891486 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:01.423491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7989436-d773-4dca-b2bc-74467b891486 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:01.424915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7989436-d773-4dca-b2bc-74467b891486 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:01.443163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 312/624] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:01.443163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f05ee-c4da-41a1-9387-1411c84e3b3b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.443163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f05ee-c4da-41a1-9387-1411c84e3b3b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.443163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f05ee-c4da-41a1-9387-1411c84e3b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.458482 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6e7f05ee-c4da-41a1-9387-1411c84e3b3b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:01.459990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6e7f05ee-c4da-41a1-9387-1411c84e3b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:01.470682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:01.497045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f1572-3364-47c2-97c0-78bb8641bcbe None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:01.540210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 313/625] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:01.546126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f05ee-c4da-41a1-9387-1411c84e3b3b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:01.547319 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8908b1a-528d-4076-9b76-30efc64f5e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:01.548343 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8908b1a-528d-4076-9b76-30efc64f5e7b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.548828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8908b1a-528d-4076-9b76-30efc64f5e7b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.549324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8908b1a-528d-4076-9b76-30efc64f5e7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.557973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8908b1a-528d-4076-9b76-30efc64f5e7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:01.560883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8908b1a-528d-4076-9b76-30efc64f5e7b None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-a8908b1a-528d-4076-9b76-30efc64f5e7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:01.577694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:01.578709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:01.579708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:01.579708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:01.579708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:01.579708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:01.579708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:01.579708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:01.579708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:01.579708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:01.582686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 313/626] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:01.588540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8908b1a-528d-4076-9b76-30efc64f5e7b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:01.633672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:01.673479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f05ee-c4da-41a1-9387-1411c84e3b3b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:01.709537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a074a-03f7-409a-b345-691f0bbb84e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:01.710479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a074a-03f7-409a-b345-691f0bbb84e0 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.710860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a074a-03f7-409a-b345-691f0bbb84e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.711233 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a074a-03f7-409a-b345-691f0bbb84e0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.719276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a074a-03f7-409a-b345-691f0bbb84e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:01.720855 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a074a-03f7-409a-b345-691f0bbb84e0 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:01.725230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8908b1a-528d-4076-9b76-30efc64f5e7b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-c78a074a-03f7-409a-b345-691f0bbb84e0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:01.736524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:01.737839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:01.739033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:01.739033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:01.739033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:01.739033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:01.739033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:01.739033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:01.739033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:01.739033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:01.739033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 18:38:01.739033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:01.746642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 314/627] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:38:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:01.747878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a074a-03f7-409a-b345-691f0bbb84e0 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:01.772128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25213650-a8da-4650-9ceb-23504605ab81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:01.773643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25213650-a8da-4650-9ceb-23504605ab81 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.774358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25213650-a8da-4650-9ceb-23504605ab81 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.774797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25213650-a8da-4650-9ceb-23504605ab81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.789596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25213650-a8da-4650-9ceb-23504605ab81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-25213650-a8da-4650-9ceb-23504605ab81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:01.800270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:01.801219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:01.802115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:01.803085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 314/628] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:38:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:01.805072 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25213650-a8da-4650-9ceb-23504605ab81 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:01.815204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:01.874297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a074a-03f7-409a-b345-691f0bbb84e0 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:01.914674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25213650-a8da-4650-9ceb-23504605ab81 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:01.916660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c710f66-d993-4706-943b-5e15d58517ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:01.918423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c710f66-d993-4706-943b-5e15d58517ec None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.918872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c710f66-d993-4706-943b-5e15d58517ec None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.919332 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c710f66-d993-4706-943b-5e15d58517ec None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.920274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c710f66-d993-4706-943b-5e15d58517ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:01.920746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c710f66-d993-4706-943b-5e15d58517ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:01.922656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c710f66-d993-4706-943b-5e15d58517ec None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:01.931867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:01.941037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 315/629] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:01.941037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c710f66-d993-4706-943b-5e15d58517ec None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:01.958793 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cef83b-54ff-4129-b88e-942c8d7dcc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:01.960405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cef83b-54ff-4129-b88e-942c8d7dcc03 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.960405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cef83b-54ff-4129-b88e-942c8d7dcc03 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.960405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cef83b-54ff-4129-b88e-942c8d7dcc03 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.961069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cef83b-54ff-4129-b88e-942c8d7dcc03 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:01.961386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cef83b-54ff-4129-b88e-942c8d7dcc03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:01.962818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cef83b-54ff-4129-b88e-942c8d7dcc03 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:01.987942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 315/630] 10.222.0.81 () {56 vars in 1137 bytes} [Thu Jun 16 18:38:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:01.989592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e9585-9c13-4902-8619-502289daac1b None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:01.990221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e9585-9c13-4902-8619-502289daac1b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:01.990779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e9585-9c13-4902-8619-502289daac1b None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:01.996644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 316/631] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:01.996644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e9585-9c13-4902-8619-502289daac1b None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:02.052590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c710f66-d993-4706-943b-5e15d58517ec None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:02.057176 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:02.089376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd748e04-bca0-48ad-ad89-b675378cc32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:02.090623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd748e04-bca0-48ad-ad89-b675378cc32e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:02.090910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd748e04-bca0-48ad-ad89-b675378cc32e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:02.091187 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd748e04-bca0-48ad-ad89-b675378cc32e None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:02.091917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd748e04-bca0-48ad-ad89-b675378cc32e None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:02.092229 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd748e04-bca0-48ad-ad89-b675378cc32e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:02.093628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd748e04-bca0-48ad-ad89-b675378cc32e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:02.097853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e9585-9c13-4902-8619-502289daac1b None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:02.101608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 316/632] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:38:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:02.102396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd748e04-bca0-48ad-ad89-b675378cc32e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:02.135652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac4056-2449-42c1-a843-5dc417cc9cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:02.137053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac4056-2449-42c1-a843-5dc417cc9cd3 None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:02.137371 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac4056-2449-42c1-a843-5dc417cc9cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:02.137643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac4056-2449-42c1-a843-5dc417cc9cd3 None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/b90f65f1f04b416286142c081c92a02d/roles/9718ec052fba437f923e8b15e08d7bed` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:02.150567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:02.152027 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac4056-2449-42c1-a843-5dc417cc9cd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=b90f65f1f04b416286142c081c92a02d, role_id=9718ec052fba437f923e8b15e08d7bed)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:02.154038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac4056-2449-42c1-a843-5dc417cc9cd3 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:02.204349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd748e04-bca0-48ad-ad89-b675378cc32e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:02.225206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 317/633] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:38:01 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/b90f65f1f04b416286142c081c92a02d/roles/9718ec052fba437f923e8b15e08d7bed => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:38:02.234943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4118f0-4b24-454f-9913-86a8579d88fb None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:02.237533 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4118f0-4b24-454f-9913-86a8579d88fb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:02.237533 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4118f0-4b24-454f-9913-86a8579d88fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:02.237871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cc487-18b0-4485-a46c-9ed973df7ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:02.238537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cc487-18b0-4485-a46c-9ed973df7ece None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:02.238822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cc487-18b0-4485-a46c-9ed973df7ece None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:02.239154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cc487-18b0-4485-a46c-9ed973df7ece None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:02.267362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-5a4118f0-4b24-454f-9913-86a8579d88fb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:02.269866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5a4118f0-4b24-454f-9913-86a8579d88fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:02.326172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cc487-18b0-4485-a46c-9ed973df7ece None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:02.327961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cc487-18b0-4485-a46c-9ed973df7ece None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:02.378436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 318/634] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:02.379357 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4118f0-4b24-454f-9913-86a8579d88fb None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:02.398396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:02.429944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:02.476517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 317/635] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:38:02.477668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cc487-18b0-4485-a46c-9ed973df7ece None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:02.497347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4118f0-4b24-454f-9913-86a8579d88fb None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:02.533005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294781e-6b02-480d-8ca7-f2c980281952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:02.533853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294781e-6b02-480d-8ca7-f2c980281952 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:02.534140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294781e-6b02-480d-8ca7-f2c980281952 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:02.534419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294781e-6b02-480d-8ca7-f2c980281952 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:02.550062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294781e-6b02-480d-8ca7-f2c980281952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:02.552604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294781e-6b02-480d-8ca7-f2c980281952 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-1294781e-6b02-480d-8ca7-f2c980281952 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:02.576531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:02.577630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:02.578492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:02.578492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:02.578492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:02.578492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:02.581384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 319/636] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:02.582373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294781e-6b02-480d-8ca7-f2c980281952 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:02.603485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cc487-18b0-4485-a46c-9ed973df7ece None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:02.640369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42387e4f-2a4c-4d11-854a-db214d81694c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:02.641175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42387e4f-2a4c-4d11-854a-db214d81694c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:02.641502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42387e4f-2a4c-4d11-854a-db214d81694c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:02.641835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42387e4f-2a4c-4d11-854a-db214d81694c None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:02.655306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42387e4f-2a4c-4d11-854a-db214d81694c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-42387e4f-2a4c-4d11-854a-db214d81694c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:02.666059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:02.666991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:02.667857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:02.675417 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 318/637] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:38:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:02.676236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42387e4f-2a4c-4d11-854a-db214d81694c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:02.690524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:02.707942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:02.713942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294781e-6b02-480d-8ca7-f2c980281952 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:02.751264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6323d1a-399d-45b9-8cde-69d62dd71f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:02.751937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323d1a-399d-45b9-8cde-69d62dd71f62 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:02.752225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323d1a-399d-45b9-8cde-69d62dd71f62 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:02.752509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6323d1a-399d-45b9-8cde-69d62dd71f62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:02.765888 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6323d1a-399d-45b9-8cde-69d62dd71f62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:02.767402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6323d1a-399d-45b9-8cde-69d62dd71f62 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-c6323d1a-399d-45b9-8cde-69d62dd71f62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:02.788832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:02.789928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:02.790812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:02.790812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:02.790812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:02.790812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:02.795861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 320/638] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:02.796797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6323d1a-399d-45b9-8cde-69d62dd71f62 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:02.820176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42387e4f-2a4c-4d11-854a-db214d81694c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:02.855109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e54e0-27c0-42b3-a6e1-526187a8f8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:02.856261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e54e0-27c0-42b3-a6e1-526187a8f8af None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:02.856545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e54e0-27c0-42b3-a6e1-526187a8f8af None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:02.856816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e54e0-27c0-42b3-a6e1-526187a8f8af None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:02.857560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e54e0-27c0-42b3-a6e1-526187a8f8af None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:02.857865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e54e0-27c0-42b3-a6e1-526187a8f8af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:02.859240 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e54e0-27c0-42b3-a6e1-526187a8f8af None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:02.872817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 319/639] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:02.873648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e54e0-27c0-42b3-a6e1-526187a8f8af None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:02.901186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6323d1a-399d-45b9-8cde-69d62dd71f62 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:02.940171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cccceb-ba4e-414b-b800-7c23debe81b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:02.941447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cccceb-ba4e-414b-b800-7c23debe81b0 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:02.942019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cccceb-ba4e-414b-b800-7c23debe81b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:02.942300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cccceb-ba4e-414b-b800-7c23debe81b0 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:02.943042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cccceb-ba4e-414b-b800-7c23debe81b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:02.943350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cccceb-ba4e-414b-b800-7c23debe81b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:02.944317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:02.946090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cccceb-ba4e-414b-b800-7c23debe81b0 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:02.955167 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 321/640] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:38:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:02.956752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68936b00-0333-499d-8a85-14a008c9a055 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:02.956934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68936b00-0333-499d-8a85-14a008c9a055 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:02.957167 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68936b00-0333-499d-8a85-14a008c9a055 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:02.958564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 322/641] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:02.959159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68936b00-0333-499d-8a85-14a008c9a055 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:02.974767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:02.987618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e54e0-27c0-42b3-a6e1-526187a8f8af None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:03.023578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41c7f3-3ca8-425f-a1f4-611163b54814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:03.024235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41c7f3-3ca8-425f-a1f4-611163b54814 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:03.024521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41c7f3-3ca8-425f-a1f4-611163b54814 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:03.024797 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc41c7f3-3ca8-425f-a1f4-611163b54814 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:03.025534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41c7f3-3ca8-425f-a1f4-611163b54814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:03.025839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41c7f3-3ca8-425f-a1f4-611163b54814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:03.027749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc41c7f3-3ca8-425f-a1f4-611163b54814 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:03.039100 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 320/642] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:03.040167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41c7f3-3ca8-425f-a1f4-611163b54814 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:03.071955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68936b00-0333-499d-8a85-14a008c9a055 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:03.108208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e9664-8fc3-40af-8143-e0432a47993c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:03.108918 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e9664-8fc3-40af-8143-e0432a47993c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:03.109199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e9664-8fc3-40af-8143-e0432a47993c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:03.109499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e9664-8fc3-40af-8143-e0432a47993c None admin] PATH_INFO: `/v3/users/nova` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:03.117276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e9664-8fc3-40af-8143-e0432a47993c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:03.118706 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e9664-8fc3-40af-8143-e0432a47993c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-4f0e9664-8fc3-40af-8143-e0432a47993c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:03.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:03.134819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:03.135781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:03.135781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:03.135781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:03.135781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:03.135781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:03.135781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:03.135781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:03.135781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:03.135781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:03.135781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:03.135781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 323/643] 10.222.0.81 () {56 vars in 1093 bytes} [Thu Jun 16 18:38:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:03.136917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bd021-b469-4feb-a701-d92b8fe58613 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:03.136917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bd021-b469-4feb-a701-d92b8fe58613 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:03.136917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bd021-b469-4feb-a701-d92b8fe58613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:03.150051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc41c7f3-3ca8-425f-a1f4-611163b54814 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:03.153508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-5f1bd021-b469-4feb-a701-d92b8fe58613 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:03.154676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5f1bd021-b469-4feb-a701-d92b8fe58613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:03.176730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:03.185675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b9d57-671c-4dbc-b17a-c28fad5982a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:03.190726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b9d57-671c-4dbc-b17a-c28fad5982a5 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:03.190726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b9d57-671c-4dbc-b17a-c28fad5982a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:03.190726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b9d57-671c-4dbc-b17a-c28fad5982a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:03.197304 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b9d57-671c-4dbc-b17a-c28fad5982a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:03.198742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b9d57-671c-4dbc-b17a-c28fad5982a5 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-482b9d57-671c-4dbc-b17a-c28fad5982a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:03.219020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:03.220115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:03.221328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:03.221328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:03.221328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:03.221328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:03.221328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 321/644] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:03.223587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b9d57-671c-4dbc-b17a-c28fad5982a5 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:03.228476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:03.245259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 324/645] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:03.245259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bd021-b469-4feb-a701-d92b8fe58613 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:03.338689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bd021-b469-4feb-a701-d92b8fe58613 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:03.346582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b9d57-671c-4dbc-b17a-c28fad5982a5 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:03.377177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63aa16-9ac4-44c5-ad2b-41a320f3aaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:03.377896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63aa16-9ac4-44c5-ad2b-41a320f3aaff None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:03.378177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63aa16-9ac4-44c5-ad2b-41a320f3aaff None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:03.378449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63aa16-9ac4-44c5-ad2b-41a320f3aaff None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:03.379288 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63aa16-9ac4-44c5-ad2b-41a320f3aaff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:03.379597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63aa16-9ac4-44c5-ad2b-41a320f3aaff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:03.381028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac63aa16-9ac4-44c5-ad2b-41a320f3aaff None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:03.387082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fe4db-32f9-4fde-91b8-89ff384fa722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:03.387891 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fe4db-32f9-4fde-91b8-89ff384fa722 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:03.388175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fe4db-32f9-4fde-91b8-89ff384fa722 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:03.388448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fe4db-32f9-4fde-91b8-89ff384fa722 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:03.396244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581fe4db-32f9-4fde-91b8-89ff384fa722 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:03.397715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581fe4db-32f9-4fde-91b8-89ff384fa722 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:03.400450 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 325/646] 10.222.0.81 () {56 vars in 1133 bytes} [Thu Jun 16 18:38:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:03.401365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63aa16-9ac4-44c5-ad2b-41a320f3aaff None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-581fe4db-32f9-4fde-91b8-89ff384fa722 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:03.417051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:03.418648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:03.419610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:03.419610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:03.419610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:03.419610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:03.419610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:03.419610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:03.419610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:03.419610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:03.419610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 322/647] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:03.419610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fe4db-32f9-4fde-91b8-89ff384fa722 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:03.463916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:03.466627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:03.510694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63aa16-9ac4-44c5-ad2b-41a320f3aaff None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:03.530002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fe4db-32f9-4fde-91b8-89ff384fa722 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:03.546926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a336136-b474-4605-a723-d9a10b9c038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:03.547699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a336136-b474-4605-a723-d9a10b9c038c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:03.547983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a336136-b474-4605-a723-d9a10b9c038c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:03.548253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a336136-b474-4605-a723-d9a10b9c038c None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:03.548960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a336136-b474-4605-a723-d9a10b9c038c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:03.549286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a336136-b474-4605-a723-d9a10b9c038c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:03.550659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a336136-b474-4605-a723-d9a10b9c038c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:03.563668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 326/648] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:03.564483 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a336136-b474-4605-a723-d9a10b9c038c None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:03.578580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85652624-2d61-4ef1-9195-d7f6b6ea3f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:03.580324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652624-2d61-4ef1-9195-d7f6b6ea3f29 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:03.580837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652624-2d61-4ef1-9195-d7f6b6ea3f29 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:03.581520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85652624-2d61-4ef1-9195-d7f6b6ea3f29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:03.594084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85652624-2d61-4ef1-9195-d7f6b6ea3f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-85652624-2d61-4ef1-9195-d7f6b6ea3f29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:03.601789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:03.602737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 323/649] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:38:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:03.609571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85652624-2d61-4ef1-9195-d7f6b6ea3f29 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:03.670817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a336136-b474-4605-a723-d9a10b9c038c None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:03.689479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:03.706099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ff68-d077-4b0a-be3b-8af400d952ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:03.706751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ff68-d077-4b0a-be3b-8af400d952ef None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:03.707039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ff68-d077-4b0a-be3b-8af400d952ef None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:03.707319 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ff68-d077-4b0a-be3b-8af400d952ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:03.711837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:03.715961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85652624-2d61-4ef1-9195-d7f6b6ea3f29 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:03.751103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908fa9e-c739-44cc-988c-4b53135f9c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:03.751771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908fa9e-c739-44cc-988c-4b53135f9c5b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:03.752372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908fa9e-c739-44cc-988c-4b53135f9c5b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:03.752655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908fa9e-c739-44cc-988c-4b53135f9c5b None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:03.753848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908fa9e-c739-44cc-988c-4b53135f9c5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:03.754167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908fa9e-c739-44cc-988c-4b53135f9c5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:03.756307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8908fa9e-c739-44cc-988c-4b53135f9c5b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:03.770060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 324/650] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:03.771083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908fa9e-c739-44cc-988c-4b53135f9c5b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:03.784507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564ff68-d077-4b0a-be3b-8af400d952ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:03.786527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564ff68-d077-4b0a-be3b-8af400d952ef None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:03.876949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908fa9e-c739-44cc-988c-4b53135f9c5b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:03.907018 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 327/651] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:38:03.907858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ff68-d077-4b0a-be3b-8af400d952ef None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:03.916037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc850a-27aa-42f4-a8bc-fee6d17d3079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:03.916842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc850a-27aa-42f4-a8bc-fee6d17d3079 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:03.917130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc850a-27aa-42f4-a8bc-fee6d17d3079 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:03.917439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc850a-27aa-42f4-a8bc-fee6d17d3079 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:03.925883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc850a-27aa-42f4-a8bc-fee6d17d3079 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:03.927293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc850a-27aa-42f4-a8bc-fee6d17d3079 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:03.927474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-5edc850a-27aa-42f4-a8bc-fee6d17d3079 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:03.941515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:03.942616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 325/652] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:38:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:03.943580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc850a-27aa-42f4-a8bc-fee6d17d3079 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:03.964650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:04.019221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ff68-d077-4b0a-be3b-8af400d952ef None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:04.053074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc850a-27aa-42f4-a8bc-fee6d17d3079 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:04.058782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39759f15-9b0a-4c4a-8eef-c03e0f432858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:04.059638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39759f15-9b0a-4c4a-8eef-c03e0f432858 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:04.060079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39759f15-9b0a-4c4a-8eef-c03e0f432858 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:04.060992 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39759f15-9b0a-4c4a-8eef-c03e0f432858 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:04.061920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39759f15-9b0a-4c4a-8eef-c03e0f432858 None admin] RBAC: Adding query filter params (name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:04.062756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39759f15-9b0a-4c4a-8eef-c03e0f432858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:04.064757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39759f15-9b0a-4c4a-8eef-c03e0f432858 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:04.076384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 328/653] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:38:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:04.077908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39759f15-9b0a-4c4a-8eef-c03e0f432858 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:04.087440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bb132-1f80-4213-b7d4-4ddda27d8cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:04.088104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bb132-1f80-4213-b7d4-4ddda27d8cbe None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:04.088390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bb132-1f80-4213-b7d4-4ddda27d8cbe None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:04.088665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bb132-1f80-4213-b7d4-4ddda27d8cbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:04.089514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bb132-1f80-4213-b7d4-4ddda27d8cbe None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=d17b7a7128d64b2ba548add02d3ec264) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:04.090414 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bb132-1f80-4213-b7d4-4ddda27d8cbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:04.091922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bb132-1f80-4213-b7d4-4ddda27d8cbe None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:04.105279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 326/654] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:03 2022] GET /identity/v3/role_assignments?user.id=d17b7a7128d64b2ba548add02d3ec264&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:04.105279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bb132-1f80-4213-b7d4-4ddda27d8cbe None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:04.177911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:04.201048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39759f15-9b0a-4c4a-8eef-c03e0f432858 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:04.226553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:04.241715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ead103-19a0-49b3-9e04-635080f96ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:04.242721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ead103-19a0-49b3-9e04-635080f96ba4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:04.243219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ead103-19a0-49b3-9e04-635080f96ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:04.243570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ead103-19a0-49b3-9e04-635080f96ba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:04.250771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bb132-1f80-4213-b7d4-4ddda27d8cbe None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:04.258421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ead103-19a0-49b3-9e04-635080f96ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:04.260498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ead103-19a0-49b3-9e04-635080f96ba4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-a3ead103-19a0-49b3-9e04-635080f96ba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:04.284707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:04.285729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:04.286612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:04.286612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:04.286612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:04.286612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:04.286612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 329/655] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:04.287340 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ead103-19a0-49b3-9e04-635080f96ba4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:04.295013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb6bec-e52b-4b58-9f39-005da80883cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:04.296089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb6bec-e52b-4b58-9f39-005da80883cb None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:04.296582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb6bec-e52b-4b58-9f39-005da80883cb None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:04.297073 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb6bec-e52b-4b58-9f39-005da80883cb None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:04.298177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb6bec-e52b-4b58-9f39-005da80883cb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:04.298713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb6bec-e52b-4b58-9f39-005da80883cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:04.300365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb6bec-e52b-4b58-9f39-005da80883cb None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:04.319937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 327/656] 10.222.0.81 () {56 vars in 1139 bytes} [Thu Jun 16 18:38:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:04.320868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb6bec-e52b-4b58-9f39-005da80883cb None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:04.429347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ead103-19a0-49b3-9e04-635080f96ba4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:04.446331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb6bec-e52b-4b58-9f39-005da80883cb None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:04.462188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:04.468473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c49a5f-c72b-4d01-91f4-bc36de05faa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:04.469667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c49a5f-c72b-4d01-91f4-bc36de05faa0 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:04.469962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c49a5f-c72b-4d01-91f4-bc36de05faa0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:04.473953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c49a5f-c72b-4d01-91f4-bc36de05faa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:04.481811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c49a5f-c72b-4d01-91f4-bc36de05faa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:04.482279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f97674-ae89-4a69-b143-428ba1046e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:04.483001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f97674-ae89-4a69-b143-428ba1046e62 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:04.483319 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c49a5f-c72b-4d01-91f4-bc36de05faa0 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:04.483442 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f97674-ae89-4a69-b143-428ba1046e62 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:04.483733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f97674-ae89-4a69-b143-428ba1046e62 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:04.484465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f97674-ae89-4a69-b143-428ba1046e62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:04.484782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f97674-ae89-4a69-b143-428ba1046e62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:04.486276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f97674-ae89-4a69-b143-428ba1046e62 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:04.495550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:04.501349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 328/657] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:04.501349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f97674-ae89-4a69-b143-428ba1046e62 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-14c49a5f-c72b-4d01-91f4-bc36de05faa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:04.502655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:04.503548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:04.504455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:04.504455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:04.504455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:04.504455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:04.508213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 330/658] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:04.509660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771efdf-a676-4ecf-b18d-7b1f9dcae6b1 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:04.509956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771efdf-a676-4ecf-b18d-7b1f9dcae6b1 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:04.510237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771efdf-a676-4ecf-b18d-7b1f9dcae6b1 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:04.513706 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 331/659] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:04.513706 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771efdf-a676-4ecf-b18d-7b1f9dcae6b1 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:04.610264 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f97674-ae89-4a69-b143-428ba1046e62 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:04.619913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771efdf-a676-4ecf-b18d-7b1f9dcae6b1 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:04.649495 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cde21e-2bdc-4ca5-95d2-7b97dfbc4799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:04.650302 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cde21e-2bdc-4ca5-95d2-7b97dfbc4799 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:04.651244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cde21e-2bdc-4ca5-95d2-7b97dfbc4799 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:04.651244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cde21e-2bdc-4ca5-95d2-7b97dfbc4799 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:04.658596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d00cb-acb4-4c1c-bcc3-e8bc14f99ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:04.659353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d00cb-acb4-4c1c-bcc3-e8bc14f99ca1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:04.659640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d00cb-acb4-4c1c-bcc3-e8bc14f99ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:04.659919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d00cb-acb4-4c1c-bcc3-e8bc14f99ca1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:04.667816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d00cb-acb4-4c1c-bcc3-e8bc14f99ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:04.669916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d00cb-acb4-4c1c-bcc3-e8bc14f99ca1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-127d00cb-acb4-4c1c-bcc3-e8bc14f99ca1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:04.685491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:04.686387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:04.687266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:04.687266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:04.687266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:04.687266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:04.687266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:04.687266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:04.687266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:04.687266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:04.690499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 332/660] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:04.691471 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d00cb-acb4-4c1c-bcc3-e8bc14f99ca1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:04.723893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:04.726772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cde21e-2bdc-4ca5-95d2-7b97dfbc4799 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:04.728518 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cde21e-2bdc-4ca5-95d2-7b97dfbc4799 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:04.739109 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:04.796588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d00cb-acb4-4c1c-bcc3-e8bc14f99ca1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:04.833585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c8966-3e2f-4f38-8a12-e7d3ccc89eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:04.834246 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8966-3e2f-4f38-8a12-e7d3ccc89eaa None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:04.834532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8966-3e2f-4f38-8a12-e7d3ccc89eaa None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:04.834808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8966-3e2f-4f38-8a12-e7d3ccc89eaa None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:04.835518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c8966-3e2f-4f38-8a12-e7d3ccc89eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:04.835936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c8966-3e2f-4f38-8a12-e7d3ccc89eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:04.836179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 329/661] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:38:04.837186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c8966-3e2f-4f38-8a12-e7d3ccc89eaa None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:04.837944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a739f-c2e7-4fe0-8ff3-820cb872e537 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:04.838433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a739f-c2e7-4fe0-8ff3-820cb872e537 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:04.838871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a739f-c2e7-4fe0-8ff3-820cb872e537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:04.851691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 333/662] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:04.851691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c8966-3e2f-4f38-8a12-e7d3ccc89eaa None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:04.860036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-701a739f-c2e7-4fe0-8ff3-820cb872e537 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:04.861441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-701a739f-c2e7-4fe0-8ff3-820cb872e537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:04.928457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 330/663] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:04.929618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a739f-c2e7-4fe0-8ff3-820cb872e537 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:04.963060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:04.968402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c8966-3e2f-4f38-8a12-e7d3ccc89eaa None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:04.971446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:05.002624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c81a5-5210-43bb-af43-0e500f21d3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:05.003288 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c81a5-5210-43bb-af43-0e500f21d3ae None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:05.003575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c81a5-5210-43bb-af43-0e500f21d3ae None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:05.003849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c81a5-5210-43bb-af43-0e500f21d3ae None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:05.004622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c81a5-5210-43bb-af43-0e500f21d3ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:05.004931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c81a5-5210-43bb-af43-0e500f21d3ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:05.006412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c81a5-5210-43bb-af43-0e500f21d3ae None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:05.022975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 334/664] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:05.022975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c81a5-5210-43bb-af43-0e500f21d3ae None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:05.042716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a739f-c2e7-4fe0-8ff3-820cb872e537 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:05.078068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8aac9-e615-428a-877b-d9953f09bcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:05.078957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8aac9-e615-428a-877b-d9953f09bcfa None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:05.079246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8aac9-e615-428a-877b-d9953f09bcfa None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:05.079521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8aac9-e615-428a-877b-d9953f09bcfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:05.092954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8aac9-e615-428a-877b-d9953f09bcfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:05.095409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8aac9-e615-428a-877b-d9953f09bcfa None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-ebc8aac9-e615-428a-877b-d9953f09bcfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:05.114658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:05.115587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:05.116499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:05.116499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:05.116499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:05.116499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:05.116499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 331/665] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:05.116774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8aac9-e615-428a-877b-d9953f09bcfa None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:05.133829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c81a5-5210-43bb-af43-0e500f21d3ae None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:05.168870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c8b31-bd9b-464c-b48d-dc87647e77e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:05.169756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c8b31-bd9b-464c-b48d-dc87647e77e6 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:05.170039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c8b31-bd9b-464c-b48d-dc87647e77e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:05.170313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c8b31-bd9b-464c-b48d-dc87647e77e6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:05.177603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c8b31-bd9b-464c-b48d-dc87647e77e6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:05.179038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c8b31-bd9b-464c-b48d-dc87647e77e6 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-ca6c8b31-bd9b-464c-b48d-dc87647e77e6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:05.193165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:05.194318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 335/666] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:38:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:05.195453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c8b31-bd9b-464c-b48d-dc87647e77e6 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:05.209409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:05.224574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:05.228220 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8aac9-e615-428a-877b-d9953f09bcfa None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:05.264633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e6220-13d6-4b4a-ad67-6287e5f36121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:05.265711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e6220-13d6-4b4a-ad67-6287e5f36121 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:05.266137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e6220-13d6-4b4a-ad67-6287e5f36121 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:05.266585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e6220-13d6-4b4a-ad67-6287e5f36121 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:05.277699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e6220-13d6-4b4a-ad67-6287e5f36121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-b90e6220-13d6-4b4a-ad67-6287e5f36121 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:05.288194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:05.289112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 332/667] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:38:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:05.295505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e6220-13d6-4b4a-ad67-6287e5f36121 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:05.299347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c8b31-bd9b-464c-b48d-dc87647e77e6 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:05.335534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e24557-4a1f-400b-8cff-a22c75084af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:05.336210 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e24557-4a1f-400b-8cff-a22c75084af0 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:05.336502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e24557-4a1f-400b-8cff-a22c75084af0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:05.336810 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e24557-4a1f-400b-8cff-a22c75084af0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:05.337668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e24557-4a1f-400b-8cff-a22c75084af0 None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=251ed026ce8046e98188066bf0bc40b7) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:05.337991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e24557-4a1f-400b-8cff-a22c75084af0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:05.339489 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e24557-4a1f-400b-8cff-a22c75084af0 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:05.349584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 336/668] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:05 2022] GET /identity/v3/role_assignments?user.id=251ed026ce8046e98188066bf0bc40b7&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:05.350800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e24557-4a1f-400b-8cff-a22c75084af0 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:05.435606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e6220-13d6-4b4a-ad67-6287e5f36121 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:05.475741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd71ac1-e901-48eb-a56f-78833ad9dbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:05.477001 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:05.478046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd71ac1-e901-48eb-a56f-78833ad9dbf0 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:05.478614 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd71ac1-e901-48eb-a56f-78833ad9dbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:05.479114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd71ac1-e901-48eb-a56f-78833ad9dbf0 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:05.480076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd71ac1-e901-48eb-a56f-78833ad9dbf0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:05.480619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd71ac1-e901-48eb-a56f-78833ad9dbf0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:05.482433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd71ac1-e901-48eb-a56f-78833ad9dbf0 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:05.492001 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e24557-4a1f-400b-8cff-a22c75084af0 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:05.498034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 333/669] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:05.499101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd71ac1-e901-48eb-a56f-78833ad9dbf0 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:05.502326 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:05.548537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50bd4-b2a6-4b31-895e-903cffc83456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:05.549230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50bd4-b2a6-4b31-895e-903cffc83456 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:05.549562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50bd4-b2a6-4b31-895e-903cffc83456 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:05.549838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50bd4-b2a6-4b31-895e-903cffc83456 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:05.550680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c50bd4-b2a6-4b31-895e-903cffc83456 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:05.550992 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c50bd4-b2a6-4b31-895e-903cffc83456 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:05.552855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c50bd4-b2a6-4b31-895e-903cffc83456 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:05.575412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 337/670] 10.222.0.81 () {56 vars in 1137 bytes} [Thu Jun 16 18:38:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:05.576367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50bd4-b2a6-4b31-895e-903cffc83456 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:05.635390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd71ac1-e901-48eb-a56f-78833ad9dbf0 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:05.688280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75749f-5070-4bd8-88af-787cbacd60ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:05.689586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75749f-5070-4bd8-88af-787cbacd60ae None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:05.689875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75749f-5070-4bd8-88af-787cbacd60ae None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:05.690307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f75749f-5070-4bd8-88af-787cbacd60ae None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:05.691170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75749f-5070-4bd8-88af-787cbacd60ae None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:05.691751 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75749f-5070-4bd8-88af-787cbacd60ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:05.693908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f75749f-5070-4bd8-88af-787cbacd60ae None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:05.701546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50bd4-b2a6-4b31-895e-903cffc83456 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:05.706114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 334/671] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:38:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:05.706955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75749f-5070-4bd8-88af-787cbacd60ae None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:05.752034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3d238-4ccb-4c39-bb51-97888453dd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:05.753274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3d238-4ccb-4c39-bb51-97888453dd13 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:05.753579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3d238-4ccb-4c39-bb51-97888453dd13 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:05.753861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3d238-4ccb-4c39-bb51-97888453dd13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:05.763678 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3d238-4ccb-4c39-bb51-97888453dd13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:05.765160 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3d238-4ccb-4c39-bb51-97888453dd13 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:05.776410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-a4b3d238-4ccb-4c39-bb51-97888453dd13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:05.785786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:05.786689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:05.787654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:05.787654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:05.787654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:05.787654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:05.787654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:05.787654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:05.787654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:05.787654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:05.787654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 338/672] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:05.788613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3d238-4ccb-4c39-bb51-97888453dd13 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:05.802164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:05.821498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f75749f-5070-4bd8-88af-787cbacd60ae None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:05.857772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27a78f-343b-4d3f-859e-66b1769de3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:05.858430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27a78f-343b-4d3f-859e-66b1769de3c4 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:05.858718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27a78f-343b-4d3f-859e-66b1769de3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:05.858997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27a78f-343b-4d3f-859e-66b1769de3c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:05.903412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3d238-4ccb-4c39-bb51-97888453dd13 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:05.931345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27a78f-343b-4d3f-859e-66b1769de3c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:05.932820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27a78f-343b-4d3f-859e-66b1769de3c4 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:05.954949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863b012-5a43-4400-a3e2-722cc12ee43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:05.955754 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863b012-5a43-4400-a3e2-722cc12ee43a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:05.956038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863b012-5a43-4400-a3e2-722cc12ee43a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:05.956313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863b012-5a43-4400-a3e2-722cc12ee43a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:05.969986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863b012-5a43-4400-a3e2-722cc12ee43a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:05.971458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863b012-5a43-4400-a3e2-722cc12ee43a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:05.996736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-1863b012-5a43-4400-a3e2-722cc12ee43a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:06.000449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:06.011916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:06.013921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:06.013921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:06.013921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:06.013921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:06.013921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 339/673] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:06.013921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863b012-5a43-4400-a3e2-722cc12ee43a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:06.049652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 335/674] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:38:06.050773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27a78f-343b-4d3f-859e-66b1769de3c4 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:06.082777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:06.129568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863b012-5a43-4400-a3e2-722cc12ee43a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:06.155152 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27a78f-343b-4d3f-859e-66b1769de3c4 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:06.165833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890a7ef-8087-42ec-95d3-aa54ab4d4c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:06.166666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890a7ef-8087-42ec-95d3-aa54ab4d4c6d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:06.167021 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890a7ef-8087-42ec-95d3-aa54ab4d4c6d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:06.167368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890a7ef-8087-42ec-95d3-aa54ab4d4c6d None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:06.168886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890a7ef-8087-42ec-95d3-aa54ab4d4c6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:06.169279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890a7ef-8087-42ec-95d3-aa54ab4d4c6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:06.171035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f890a7ef-8087-42ec-95d3-aa54ab4d4c6d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:06.184520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 340/675] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:06.185451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890a7ef-8087-42ec-95d3-aa54ab4d4c6d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:06.197855 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a35e8-a553-4458-9059-951c1ad84fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:06.198653 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a35e8-a553-4458-9059-951c1ad84fac None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:06.199083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a35e8-a553-4458-9059-951c1ad84fac None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:06.199507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a35e8-a553-4458-9059-951c1ad84fac None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:06.200448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a35e8-a553-4458-9059-951c1ad84fac None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:06.200860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a35e8-a553-4458-9059-951c1ad84fac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:06.202440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a35e8-a553-4458-9059-951c1ad84fac None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:06.215937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 336/676] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:06.221133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a35e8-a553-4458-9059-951c1ad84fac None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:06.253201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:06.304030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f890a7ef-8087-42ec-95d3-aa54ab4d4c6d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:06.328940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:06.358000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e5b6e-7f88-4932-b0e4-c459a5fe1eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:06.359253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e5b6e-7f88-4932-b0e4-c459a5fe1eb7 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:06.359797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e5b6e-7f88-4932-b0e4-c459a5fe1eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:06.360294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a35e8-a553-4458-9059-951c1ad84fac None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:06.360429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e5b6e-7f88-4932-b0e4-c459a5fe1eb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:06.376722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e5b6e-7f88-4932-b0e4-c459a5fe1eb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:06.378832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e5b6e-7f88-4932-b0e4-c459a5fe1eb7 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-a44e5b6e-7f88-4932-b0e4-c459a5fe1eb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:06.404892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:06.405869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:06.406756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:06.406756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:06.406756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:06.406756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:06.406756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 341/677] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:06.407494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e5b6e-7f88-4932-b0e4-c459a5fe1eb7 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:06.416128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650730f-6b0a-43ec-97bb-264bfc3cf276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:06.417274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650730f-6b0a-43ec-97bb-264bfc3cf276 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:06.417675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650730f-6b0a-43ec-97bb-264bfc3cf276 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:06.418005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650730f-6b0a-43ec-97bb-264bfc3cf276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:06.418876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650730f-6b0a-43ec-97bb-264bfc3cf276 None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=f4e05d75cd1142e18d7518b78edd88fa) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:06.419555 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650730f-6b0a-43ec-97bb-264bfc3cf276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:06.421382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650730f-6b0a-43ec-97bb-264bfc3cf276 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:06.427557 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 337/678] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:06 2022] GET /identity/v3/role_assignments?user.id=f4e05d75cd1142e18d7518b78edd88fa&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:06.430476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650730f-6b0a-43ec-97bb-264bfc3cf276 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:06.535907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:06.542107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e5b6e-7f88-4932-b0e4-c459a5fe1eb7 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:06.549590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650730f-6b0a-43ec-97bb-264bfc3cf276 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:06.589268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a13f8f-3502-4718-8b11-18ba66d2852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:06.590052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d9b07-cc44-43b8-9e58-a3458d00edcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:06.590789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a13f8f-3502-4718-8b11-18ba66d2852a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:06.591174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a13f8f-3502-4718-8b11-18ba66d2852a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:06.591365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d9b07-cc44-43b8-9e58-a3458d00edcb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:06.591634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a13f8f-3502-4718-8b11-18ba66d2852a None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:06.591733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d9b07-cc44-43b8-9e58-a3458d00edcb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:06.591915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d9b07-cc44-43b8-9e58-a3458d00edcb None admin] PATH_INFO: `/v3/users/swift` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:06.592611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a13f8f-3502-4718-8b11-18ba66d2852a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:06.593015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a13f8f-3502-4718-8b11-18ba66d2852a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:06.594740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a13f8f-3502-4718-8b11-18ba66d2852a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:06.600909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d9b07-cc44-43b8-9e58-a3458d00edcb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:06.602546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d9b07-cc44-43b8-9e58-a3458d00edcb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:06.608178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 338/679] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:06.610092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919aa3d-44fa-4f53-b0c0-6aacfe64d237 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:06.610092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919aa3d-44fa-4f53-b0c0-6aacfe64d237 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:06.610092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2919aa3d-44fa-4f53-b0c0-6aacfe64d237 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:06.611382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 339/680] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:06.616836 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919aa3d-44fa-4f53-b0c0-6aacfe64d237 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-dc3d9b07-cc44-43b8-9e58-a3458d00edcb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:06.622282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:06.623211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:06.624799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:06.624799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:06.624799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:06.624799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:06.624799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:06.624799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:06.624799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:06.624799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:06.624799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:38:06.624799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:06.624799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 342/681] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:06.625498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349eef3-8317-404b-8169-61fbf1b0453a None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:06.625498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349eef3-8317-404b-8169-61fbf1b0453a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:06.625644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349eef3-8317-404b-8169-61fbf1b0453a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:06.646333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9349eef3-8317-404b-8169-61fbf1b0453a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:06.646760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9349eef3-8317-404b-8169-61fbf1b0453a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:06.680924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:06.743010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2919aa3d-44fa-4f53-b0c0-6aacfe64d237 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:06.744184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 343/682] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:06.746123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349eef3-8317-404b-8169-61fbf1b0453a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:06.789522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fe085-4b09-4883-a7b6-1670b919cf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:06.790707 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fe085-4b09-4883-a7b6-1670b919cf07 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:06.791175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fe085-4b09-4883-a7b6-1670b919cf07 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:06.791631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610fe085-4b09-4883-a7b6-1670b919cf07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:06.799253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fe085-4b09-4883-a7b6-1670b919cf07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:06.800941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610fe085-4b09-4883-a7b6-1670b919cf07 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-610fe085-4b09-4883-a7b6-1670b919cf07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:06.823637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:06.825147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:06.826546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:06.826546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:06.826546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:06.826546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:06.826546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:06.826546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:06.826546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:06.826546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:06.827372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 340/683] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:06.829206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fe085-4b09-4883-a7b6-1670b919cf07 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:06.849699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:06.862806 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349eef3-8317-404b-8169-61fbf1b0453a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:06.900351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4a834-710b-4a0e-a6d4-bc4ac2ff90fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:06.901026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4a834-710b-4a0e-a6d4-bc4ac2ff90fb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:06.901349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4a834-710b-4a0e-a6d4-bc4ac2ff90fb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:06.901628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4a834-710b-4a0e-a6d4-bc4ac2ff90fb None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:06.902473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4a834-710b-4a0e-a6d4-bc4ac2ff90fb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:06.902796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4a834-710b-4a0e-a6d4-bc4ac2ff90fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:06.904234 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4a834-710b-4a0e-a6d4-bc4ac2ff90fb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:06.923227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 344/684] 10.222.0.81 () {56 vars in 1135 bytes} [Thu Jun 16 18:38:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:06.924053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4a834-710b-4a0e-a6d4-bc4ac2ff90fb None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:06.943630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:06.967966 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610fe085-4b09-4883-a7b6-1670b919cf07 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:07.014467 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8868c49-7743-45b7-b037-aacc38cd8821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:07.015796 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8868c49-7743-45b7-b037-aacc38cd8821 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:07.016236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8868c49-7743-45b7-b037-aacc38cd8821 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:07.016679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8868c49-7743-45b7-b037-aacc38cd8821 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:07.029915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8868c49-7743-45b7-b037-aacc38cd8821 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:07.031562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4a834-710b-4a0e-a6d4-bc4ac2ff90fb None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:07.031841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8868c49-7743-45b7-b037-aacc38cd8821 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-c8868c49-7743-45b7-b037-aacc38cd8821 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:07.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:07.058904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:07.060318 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:07.060318 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:07.060318 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:07.060318 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:07.062601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 341/685] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:07.063562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8868c49-7743-45b7-b037-aacc38cd8821 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:07.089044 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8c8ce-33b0-4e44-b052-e147b7c5b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:07.089922 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8c8ce-33b0-4e44-b052-e147b7c5b048 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:07.090386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8c8ce-33b0-4e44-b052-e147b7c5b048 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:07.090774 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8c8ce-33b0-4e44-b052-e147b7c5b048 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:07.092104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8c8ce-33b0-4e44-b052-e147b7c5b048 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:07.092659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8c8ce-33b0-4e44-b052-e147b7c5b048 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:07.094476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8c8ce-33b0-4e44-b052-e147b7c5b048 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:07.119436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 345/686] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:07.120846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8c8ce-33b0-4e44-b052-e147b7c5b048 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:07.157701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:07.173421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8868c49-7743-45b7-b037-aacc38cd8821 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:07.217704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3060a-3b9f-4dac-9081-59c9f45c2c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:07.218987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:07.219730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3060a-3b9f-4dac-9081-59c9f45c2c2c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:07.220345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3060a-3b9f-4dac-9081-59c9f45c2c2c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:07.220639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3060a-3b9f-4dac-9081-59c9f45c2c2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:07.241886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8c8ce-33b0-4e44-b052-e147b7c5b048 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:07.280053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bf03b-9190-4d35-8bc5-0c50c8fa8870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:07.280894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bf03b-9190-4d35-8bc5-0c50c8fa8870 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:07.281315 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bf03b-9190-4d35-8bc5-0c50c8fa8870 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:07.281696 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bf03b-9190-4d35-8bc5-0c50c8fa8870 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:07.282609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bf03b-9190-4d35-8bc5-0c50c8fa8870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:07.283056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bf03b-9190-4d35-8bc5-0c50c8fa8870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:07.284512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bf03b-9190-4d35-8bc5-0c50c8fa8870 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:07.301584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 346/687] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:07.302638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bf03b-9190-4d35-8bc5-0c50c8fa8870 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:07.308368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3060a-3b9f-4dac-9081-59c9f45c2c2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:07.310412 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3060a-3b9f-4dac-9081-59c9f45c2c2c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:07.385618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:07.430915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 342/688] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:38:07.432169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3060a-3b9f-4dac-9081-59c9f45c2c2c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:07.451810 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bf03b-9190-4d35-8bc5-0c50c8fa8870 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:07.498672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ccae5-2074-45c9-92f1-578b3a64f0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:07.499331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ccae5-2074-45c9-92f1-578b3a64f0a9 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:07.499610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ccae5-2074-45c9-92f1-578b3a64f0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:07.499884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ccae5-2074-45c9-92f1-578b3a64f0a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:07.500904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ccae5-2074-45c9-92f1-578b3a64f0a9 None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=9c82335b7ec74fb3a1508a2fc76335ab) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:07.501217 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ccae5-2074-45c9-92f1-578b3a64f0a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:07.503147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ccae5-2074-45c9-92f1-578b3a64f0a9 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:07.511100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 347/689] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:07 2022] GET /identity/v3/role_assignments?user.id=9c82335b7ec74fb3a1508a2fc76335ab&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:07.512110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ccae5-2074-45c9-92f1-578b3a64f0a9 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:07.515742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:07.567518 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3060a-3b9f-4dac-9081-59c9f45c2c2c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:07.634540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ccae5-2074-45c9-92f1-578b3a64f0a9 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:07.663137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236c58b-42d3-43b7-8bb3-71799aa0ec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:07.664193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236c58b-42d3-43b7-8bb3-71799aa0ec37 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:07.664681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236c58b-42d3-43b7-8bb3-71799aa0ec37 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:07.665156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236c58b-42d3-43b7-8bb3-71799aa0ec37 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:07.666084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236c58b-42d3-43b7-8bb3-71799aa0ec37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:07.667746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236c58b-42d3-43b7-8bb3-71799aa0ec37 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:07.676115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8462c6f-cb4a-4b77-9f34-f86adc287e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:07.676924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8462c6f-cb4a-4b77-9f34-f86adc287e7f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:07.677202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8462c6f-cb4a-4b77-9f34-f86adc287e7f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:07.677505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8462c6f-cb4a-4b77-9f34-f86adc287e7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:07.689849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8462c6f-cb4a-4b77-9f34-f86adc287e7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:07.691799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8462c6f-cb4a-4b77-9f34-f86adc287e7f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-f8462c6f-cb4a-4b77-9f34-f86adc287e7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:07.712114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:07.713099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:07.714061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:07.714061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:07.714061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:07.714061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:07.714061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 348/690] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:07.722454 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8462c6f-cb4a-4b77-9f34-f86adc287e7f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:07.787357 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 343/691] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:38:07 2022] POST /identity/v3/users => generated 254 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:07.799581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:07.832410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312834f-ac67-46d5-99ea-fca44293aff8 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:07.832706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312834f-ac67-46d5-99ea-fca44293aff8 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:07.832990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312834f-ac67-46d5-99ea-fca44293aff8 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:07.834630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 344/692] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:07.842587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96e1a6-3da7-4a79-948a-dd7961edfe25 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:07.842967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96e1a6-3da7-4a79-948a-dd7961edfe25 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:07.843158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96e1a6-3da7-4a79-948a-dd7961edfe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:07.855935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8462c6f-cb4a-4b77-9f34-f86adc287e7f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:07.886420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-7a96e1a6-3da7-4a79-948a-dd7961edfe25 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:07.888146 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-7a96e1a6-3da7-4a79-948a-dd7961edfe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:07.898409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89a097-c0ce-45bf-be48-ed416bed9e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:07.899924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89a097-c0ce-45bf-be48-ed416bed9e41 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:07.900412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89a097-c0ce-45bf-be48-ed416bed9e41 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:07.900925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89a097-c0ce-45bf-be48-ed416bed9e41 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:07.901526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:07.903949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89a097-c0ce-45bf-be48-ed416bed9e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:07.904532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89a097-c0ce-45bf-be48-ed416bed9e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:07.906478 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89a097-c0ce-45bf-be48-ed416bed9e41 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:07.927194 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 349/693] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:07.927194 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89a097-c0ce-45bf-be48-ed416bed9e41 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:07.973835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 345/694] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:08.002120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96e1a6-3da7-4a79-948a-dd7961edfe25 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:08.063951 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89a097-c0ce-45bf-be48-ed416bed9e41 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:08.079357 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:08.102470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69e77c-a02e-4376-9de9-a6505266872f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:08.103384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69e77c-a02e-4376-9de9-a6505266872f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:08.103675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69e77c-a02e-4376-9de9-a6505266872f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:08.103955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69e77c-a02e-4376-9de9-a6505266872f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:08.111410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96e1a6-3da7-4a79-948a-dd7961edfe25 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:08.113069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69e77c-a02e-4376-9de9-a6505266872f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:08.115893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69e77c-a02e-4376-9de9-a6505266872f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-fc69e77c-a02e-4376-9de9-a6505266872f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:08.134445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:08.135411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:08.136764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:08.136764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:08.136764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:08.136764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:08.136764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:08.136764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:08.136764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:08.136764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:08.136764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 350/695] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:08.143131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69e77c-a02e-4376-9de9-a6505266872f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:08.164675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bb884-9287-403a-b235-a5976c5097cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:08.165880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bb884-9287-403a-b235-a5976c5097cf None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:08.166382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bb884-9287-403a-b235-a5976c5097cf None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:08.166863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6bb884-9287-403a-b235-a5976c5097cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:08.178638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6bb884-9287-403a-b235-a5976c5097cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-4b6bb884-9287-403a-b235-a5976c5097cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:08.187347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:08.188323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:08.189162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:08.195116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 346/696] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:08.198192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bb884-9287-403a-b235-a5976c5097cf None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:08.208714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:08.255523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69e77c-a02e-4376-9de9-a6505266872f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:08.291670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498896b-0393-41b7-8017-e0b1ac78546e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:08.292336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498896b-0393-41b7-8017-e0b1ac78546e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:08.292623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498896b-0393-41b7-8017-e0b1ac78546e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:08.292902 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498896b-0393-41b7-8017-e0b1ac78546e None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:08.293996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498896b-0393-41b7-8017-e0b1ac78546e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:08.294107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498896b-0393-41b7-8017-e0b1ac78546e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:08.295484 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498896b-0393-41b7-8017-e0b1ac78546e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:08.306902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6bb884-9287-403a-b235-a5976c5097cf None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:08.308582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 351/697] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:08.314885 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498896b-0393-41b7-8017-e0b1ac78546e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:08.348030 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d15d68-e94b-454a-84d0-8b9750c3e2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:08.349075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15d68-e94b-454a-84d0-8b9750c3e2d9 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:08.349655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15d68-e94b-454a-84d0-8b9750c3e2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:08.349947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15d68-e94b-454a-84d0-8b9750c3e2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:08.350523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:08.350728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d15d68-e94b-454a-84d0-8b9750c3e2d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:08.351038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d15d68-e94b-454a-84d0-8b9750c3e2d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:08.352456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d15d68-e94b-454a-84d0-8b9750c3e2d9 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:08.373173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 347/698] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:38:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:08.381230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d15d68-e94b-454a-84d0-8b9750c3e2d9 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:08.481749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498896b-0393-41b7-8017-e0b1ac78546e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:08.513972 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:08.555538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e14d6-07ee-420f-b591-0be5b970cee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:08.556531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e14d6-07ee-420f-b591-0be5b970cee1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:08.557006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e14d6-07ee-420f-b591-0be5b970cee1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:08.557509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e14d6-07ee-420f-b591-0be5b970cee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:08.558510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e14d6-07ee-420f-b591-0be5b970cee1 None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=b90f65f1f04b416286142c081c92a02d) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:08.559588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e14d6-07ee-420f-b591-0be5b970cee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:08.563342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e14d6-07ee-420f-b591-0be5b970cee1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:08.574219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 352/699] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:08 2022] GET /identity/v3/role_assignments?user.id=b90f65f1f04b416286142c081c92a02d&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:08.582865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d15d68-e94b-454a-84d0-8b9750c3e2d9 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:08.630870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246a359-e0ce-4452-8178-eff6a317d1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:08.632011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246a359-e0ce-4452-8178-eff6a317d1ae None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:08.632512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246a359-e0ce-4452-8178-eff6a317d1ae None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:08.633059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246a359-e0ce-4452-8178-eff6a317d1ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:08.645950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246a359-e0ce-4452-8178-eff6a317d1ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:08.647778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246a359-e0ce-4452-8178-eff6a317d1ae None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-8246a359-e0ce-4452-8178-eff6a317d1ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:08.666803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:08.674120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:08.675007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:08.675007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:08.675007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:08.675007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:08.675007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 348/700] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:08.677908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e14d6-07ee-420f-b591-0be5b970cee1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:08.800568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:08.804789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e14d6-07ee-420f-b591-0be5b970cee1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:08.843791 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a1fdc-c0ba-4cf1-b278-d17578aadc9b None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:08.844118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a1fdc-c0ba-4cf1-b278-d17578aadc9b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:08.844335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a1fdc-c0ba-4cf1-b278-d17578aadc9b None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:08.845301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e009e5-7d1e-4fab-a361-e78df0991dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:08.846275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e009e5-7d1e-4fab-a361-e78df0991dd1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:08.846794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e009e5-7d1e-4fab-a361-e78df0991dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:08.847336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e009e5-7d1e-4fab-a361-e78df0991dd1 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:08.848274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e009e5-7d1e-4fab-a361-e78df0991dd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:08.848791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e009e5-7d1e-4fab-a361-e78df0991dd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:08.849360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 349/701] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:08.850753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e009e5-7d1e-4fab-a361-e78df0991dd1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:08.851083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32089d9e-332b-47f2-b2df-38817488f638 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:08.851369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32089d9e-332b-47f2-b2df-38817488f638 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:08.851645 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32089d9e-332b-47f2-b2df-38817488f638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:08.865290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 353/702] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:08.872921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e009e5-7d1e-4fab-a361-e78df0991dd1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:08.877282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-32089d9e-332b-47f2-b2df-38817488f638 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:08.878496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-32089d9e-332b-47f2-b2df-38817488f638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:08.957467 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 350/703] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:38:08.977104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32089d9e-332b-47f2-b2df-38817488f638 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:08.982662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e009e5-7d1e-4fab-a361-e78df0991dd1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:09.018949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9d75e-6254-414e-99a3-f7f9e2a8c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:09.020154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9d75e-6254-414e-99a3-f7f9e2a8c550 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:09.020645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9d75e-6254-414e-99a3-f7f9e2a8c550 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:09.021216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9d75e-6254-414e-99a3-f7f9e2a8c550 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:09.030268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:09.030960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9d75e-6254-414e-99a3-f7f9e2a8c550 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:09.032612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9d75e-6254-414e-99a3-f7f9e2a8c550 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-75d9d75e-6254-414e-99a3-f7f9e2a8c550 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:09.049274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:09.050373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:09.051342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:09.051342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:09.051342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:09.051342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:09.051342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:09.051342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:09.051342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:09.051342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:09.051342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:09.051342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:09.054199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 354/704] 10.222.0.81 () {56 vars in 1093 bytes} [Thu Jun 16 18:38:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:09.059838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9d75e-6254-414e-99a3-f7f9e2a8c550 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:09.080984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:09.081424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32089d9e-332b-47f2-b2df-38817488f638 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:09.119513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c5100-3c14-4f98-bebf-59be0a7f1c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:09.120407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c5100-3c14-4f98-bebf-59be0a7f1c0c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:09.120683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c5100-3c14-4f98-bebf-59be0a7f1c0c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:09.120954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c5100-3c14-4f98-bebf-59be0a7f1c0c None admin] PATH_INFO: `/v3/services/network` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:09.121682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c5100-3c14-4f98-bebf-59be0a7f1c0c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:09.123189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c5100-3c14-4f98-bebf-59be0a7f1c0c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-8e3c5100-3c14-4f98-bebf-59be0a7f1c0c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:09.143661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:09.144818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:38:09.145708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:09.148880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 351/705] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:08 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:09.151621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c5100-3c14-4f98-bebf-59be0a7f1c0c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:09.186124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9d75e-6254-414e-99a3-f7f9e2a8c550 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:09.223828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9745893-819b-4e33-a4c6-cc67d8a4808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:09.224843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9745893-819b-4e33-a4c6-cc67d8a4808d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:09.225395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9745893-819b-4e33-a4c6-cc67d8a4808d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:09.225996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9745893-819b-4e33-a4c6-cc67d8a4808d None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:09.227150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9745893-819b-4e33-a4c6-cc67d8a4808d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:09.227680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9745893-819b-4e33-a4c6-cc67d8a4808d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:09.229336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9745893-819b-4e33-a4c6-cc67d8a4808d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:09.250958 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 355/706] 10.222.0.81 () {56 vars in 1133 bytes} [Thu Jun 16 18:38:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:09.256687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9745893-819b-4e33-a4c6-cc67d8a4808d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:09.258190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c5100-3c14-4f98-bebf-59be0a7f1c0c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:09.275892 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:09.294423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760a929-499c-42bf-a6f8-525ea34b1a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:09.295092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760a929-499c-42bf-a6f8-525ea34b1a76 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:09.295374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760a929-499c-42bf-a6f8-525ea34b1a76 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:09.295644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760a929-499c-42bf-a6f8-525ea34b1a76 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:09.296417 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760a929-499c-42bf-a6f8-525ea34b1a76 None admin] RBAC: Adding query filter params (name=network) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:09.296791 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760a929-499c-42bf-a6f8-525ea34b1a76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:09.298249 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760a929-499c-42bf-a6f8-525ea34b1a76 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:09.307057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 352/707] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:09.308877 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760a929-499c-42bf-a6f8-525ea34b1a76 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:09.418210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760a929-499c-42bf-a6f8-525ea34b1a76 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:09.427221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:09.453593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb519d6e-62b3-4ed0-bfba-0ffa545b1d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:09.454266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb519d6e-62b3-4ed0-bfba-0ffa545b1d16 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:09.454553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb519d6e-62b3-4ed0-bfba-0ffa545b1d16 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:09.454830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb519d6e-62b3-4ed0-bfba-0ffa545b1d16 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:09.455526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb519d6e-62b3-4ed0-bfba-0ffa545b1d16 None admin] RBAC: Adding query filter params (type=network) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:09.455836 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb519d6e-62b3-4ed0-bfba-0ffa545b1d16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:09.457406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb519d6e-62b3-4ed0-bfba-0ffa545b1d16 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:09.469544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 353/708] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:09 2022] GET /identity/v3/services?type=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:09.490640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9745893-819b-4e33-a4c6-cc67d8a4808d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:09.543253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5cce1-bdfd-4f53-a846-548f904bef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:09.543975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5cce1-bdfd-4f53-a846-548f904bef34 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:09.544265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5cce1-bdfd-4f53-a846-548f904bef34 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:09.544542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5cce1-bdfd-4f53-a846-548f904bef34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:09.631497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5cce1-bdfd-4f53-a846-548f904bef34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:09.632744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5cce1-bdfd-4f53-a846-548f904bef34 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:09.791389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:09.822957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 356/709] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:38:09.829234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb519d6e-62b3-4ed0-bfba-0ffa545b1d16 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:09.878112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:09.937211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb519d6e-62b3-4ed0-bfba-0ffa545b1d16 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:09.972838 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e853c2e-5336-42cc-992b-f36b83eac703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:09.974035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e853c2e-5336-42cc-992b-f36b83eac703 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:09.974517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e853c2e-5336-42cc-992b-f36b83eac703 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:09.975001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e853c2e-5336-42cc-992b-f36b83eac703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:09.986216 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e853c2e-5336-42cc-992b-f36b83eac703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:09.987940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e853c2e-5336-42cc-992b-f36b83eac703 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-9e853c2e-5336-42cc-992b-f36b83eac703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:10.006265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:10.007158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:10.008036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:10.008036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:10.008036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:10.008036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:10.010283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 354/710] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:10.017283 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5cce1-bdfd-4f53-a846-548f904bef34 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:10.159862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5cce1-bdfd-4f53-a846-548f904bef34 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:10.194460 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf980f9-7c6e-42d3-b048-3a2740b4974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:10.195043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf980f9-7c6e-42d3-b048-3a2740b4974a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:10.195333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf980f9-7c6e-42d3-b048-3a2740b4974a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:10.195612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf980f9-7c6e-42d3-b048-3a2740b4974a None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:10.196335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf980f9-7c6e-42d3-b048-3a2740b4974a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:10.196646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf980f9-7c6e-42d3-b048-3a2740b4974a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:10.198039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf980f9-7c6e-42d3-b048-3a2740b4974a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:10.214653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 357/711] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:10.220655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e853c2e-5336-42cc-992b-f36b83eac703 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:10.327741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:10.334458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e853c2e-5336-42cc-992b-f36b83eac703 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:10.370772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba48739-922f-4251-887b-545809dbbf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:10.371574 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba48739-922f-4251-887b-545809dbbf6a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:10.371877 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba48739-922f-4251-887b-545809dbbf6a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:10.372145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba48739-922f-4251-887b-545809dbbf6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:10.379134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba48739-922f-4251-887b-545809dbbf6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:10.380576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba48739-922f-4251-887b-545809dbbf6a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-5ba48739-922f-4251-887b-545809dbbf6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:10.395018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:10.395922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:10.396920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:10.396920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:10.396920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:10.396920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:10.396920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:10.396920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:10.396920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:10.396920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:10.396920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 355/712] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:10.404811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf980f9-7c6e-42d3-b048-3a2740b4974a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:10.469843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:10.559650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf980f9-7c6e-42d3-b048-3a2740b4974a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:10.616088 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2aff35-7e93-4f7d-bf71-a9f9683af91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:10.616777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aff35-7e93-4f7d-bf71-a9f9683af91a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:10.617063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aff35-7e93-4f7d-bf71-a9f9683af91a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:10.617366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2aff35-7e93-4f7d-bf71-a9f9683af91a None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:10.618172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aff35-7e93-4f7d-bf71-a9f9683af91a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:10.618486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aff35-7e93-4f7d-bf71-a9f9683af91a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:10.620032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2aff35-7e93-4f7d-bf71-a9f9683af91a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:10.637506 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 358/713] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:10.643547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba48739-922f-4251-887b-545809dbbf6a None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:10.682438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea04d26-e926-4f6d-b745-bd6fe908a3be None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:10.682844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea04d26-e926-4f6d-b745-bd6fe908a3be None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:10.683128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea04d26-e926-4f6d-b745-bd6fe908a3be None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:10.684783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 359/714] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:10.687056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371c708-a41d-4d0b-b4c7-5da6fbd3e479 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:10.687346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371c708-a41d-4d0b-b4c7-5da6fbd3e479 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:10.687628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371c708-a41d-4d0b-b4c7-5da6fbd3e479 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:10.689357 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 360/715] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:10.690724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70ebd6-4b57-4e38-9487-1e527c71a63c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:10.691259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70ebd6-4b57-4e38-9487-1e527c71a63c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:10.691779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70ebd6-4b57-4e38-9487-1e527c71a63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:10.709336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-8f70ebd6-4b57-4e38-9487-1e527c71a63c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:10.710816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8f70ebd6-4b57-4e38-9487-1e527c71a63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:10.767375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba48739-922f-4251-887b-545809dbbf6a None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:10.797413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 361/716] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 18:38:10.799056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ace6-23d8-4e7a-ad6e-b0f643874528 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:10.799512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ace6-23d8-4e7a-ad6e-b0f643874528 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:10.799952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1ace6-23d8-4e7a-ad6e-b0f643874528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:10.817081 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-82f1ace6-23d8-4e7a-ad6e-b0f643874528 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:10.818758 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-82f1ace6-23d8-4e7a-ad6e-b0f643874528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:10.822511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb06f5-ce8d-4ece-b0c5-ce14171ed06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:10.823259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb06f5-ce8d-4ece-b0c5-ce14171ed06e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:10.823623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb06f5-ce8d-4ece-b0c5-ce14171ed06e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:10.823976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb06f5-ce8d-4ece-b0c5-ce14171ed06e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:10.825441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb06f5-ce8d-4ece-b0c5-ce14171ed06e None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=251ed026ce8046e98188066bf0bc40b7) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:10.825832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb06f5-ce8d-4ece-b0c5-ce14171ed06e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:10.827345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb06f5-ce8d-4ece-b0c5-ce14171ed06e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:10.838228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 356/717] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:10 2022] GET /identity/v3/role_assignments?user.id=251ed026ce8046e98188066bf0bc40b7&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:10.839131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb06f5-ce8d-4ece-b0c5-ce14171ed06e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:10.843827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:10.898171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 362/718] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:10.899040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:10.940447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1ace6-23d8-4e7a-ad6e-b0f643874528 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:11.051929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb06f5-ce8d-4ece-b0c5-ce14171ed06e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:11.088691 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51e5c-d864-4436-9039-39c343ca0e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:11.089488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51e5c-d864-4436-9039-39c343ca0e00 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:11.089775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51e5c-d864-4436-9039-39c343ca0e00 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:11.090054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51e5c-d864-4436-9039-39c343ca0e00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:11.094050 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1ace6-23d8-4e7a-ad6e-b0f643874528 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:11.101840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c51e5c-d864-4436-9039-39c343ca0e00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-95c51e5c-d864-4436-9039-39c343ca0e00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:11.109897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:11.110929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:11.111834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 357/719] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:38:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:11.117503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51e5c-d864-4436-9039-39c343ca0e00 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:11.212160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:11.236084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51e5c-d864-4436-9039-39c343ca0e00 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:11.250707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92ae31-96a0-48c9-9163-1db96ad755e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:11.251854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ae31-96a0-48c9-9163-1db96ad755e4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:11.252342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ae31-96a0-48c9-9163-1db96ad755e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:11.252812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92ae31-96a0-48c9-9163-1db96ad755e4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:11.253723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92ae31-96a0-48c9-9163-1db96ad755e4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:11.255291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92ae31-96a0-48c9-9163-1db96ad755e4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-1f92ae31-96a0-48c9-9163-1db96ad755e4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:11.270219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:11.271230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:38:11.272076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:11.274661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c557e9-20f8-4c72-9e95-707324691558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:11.275224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 363/720] 10.222.0.81 () {56 vars in 1129 bytes} [Thu Jun 16 18:38:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:11.275462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c557e9-20f8-4c72-9e95-707324691558 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:11.275826 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c557e9-20f8-4c72-9e95-707324691558 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:11.276173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c557e9-20f8-4c72-9e95-707324691558 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:11.276983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c557e9-20f8-4c72-9e95-707324691558 None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:11.281540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c557e9-20f8-4c72-9e95-707324691558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:11.283239 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c557e9-20f8-4c72-9e95-707324691558 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:11.285541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92ae31-96a0-48c9-9163-1db96ad755e4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:11.293766 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 358/721] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:38:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:11.299923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c557e9-20f8-4c72-9e95-707324691558 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:11.308860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:11.410669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c557e9-20f8-4c72-9e95-707324691558 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:11.444064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92ae31-96a0-48c9-9163-1db96ad755e4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:11.452986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df67190-e947-4285-bae6-069e0c3d7946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:11.454120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df67190-e947-4285-bae6-069e0c3d7946 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:11.454610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df67190-e947-4285-bae6-069e0c3d7946 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:11.455086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df67190-e947-4285-bae6-069e0c3d7946 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:11.464466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:11.468190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df67190-e947-4285-bae6-069e0c3d7946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:11.469956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df67190-e947-4285-bae6-069e0c3d7946 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-6df67190-e947-4285-bae6-069e0c3d7946 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:11.488492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:11.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:11.490362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:11.490362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:11.490362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:11.490362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:11.492601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 359/722] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:11.498646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df67190-e947-4285-bae6-069e0c3d7946 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:11.513956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e96ec6-f37c-488e-b36d-8e7c0c28f4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:11.514723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e96ec6-f37c-488e-b36d-8e7c0c28f4b4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:11.514925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e96ec6-f37c-488e-b36d-8e7c0c28f4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:11.515188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e96ec6-f37c-488e-b36d-8e7c0c28f4b4 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:11.515900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e96ec6-f37c-488e-b36d-8e7c0c28f4b4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:11.516208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e96ec6-f37c-488e-b36d-8e7c0c28f4b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:11.517610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e96ec6-f37c-488e-b36d-8e7c0c28f4b4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:11.522605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 364/723] 10.222.0.81 () {56 vars in 1124 bytes} [Thu Jun 16 18:38:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:11.537591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e96ec6-f37c-488e-b36d-8e7c0c28f4b4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:11.619161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df67190-e947-4285-bae6-069e0c3d7946 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:11.657661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b826a52e-75b7-4a08-b16c-b4ca6cb75c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:11.658664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826a52e-75b7-4a08-b16c-b4ca6cb75c5f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:11.660041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826a52e-75b7-4a08-b16c-b4ca6cb75c5f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:11.660041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b826a52e-75b7-4a08-b16c-b4ca6cb75c5f None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:11.660426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826a52e-75b7-4a08-b16c-b4ca6cb75c5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:11.660912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826a52e-75b7-4a08-b16c-b4ca6cb75c5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:11.662527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b826a52e-75b7-4a08-b16c-b4ca6cb75c5f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:11.675703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 360/724] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:11.681672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:11.681937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e10ab-6ff4-47ec-b54d-6ee723adaecd None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:11.682472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e10ab-6ff4-47ec-b54d-6ee723adaecd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:11.682971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e10ab-6ff4-47ec-b54d-6ee723adaecd None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:11.684828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 361/725] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:11.687571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e10ab-6ff4-47ec-b54d-6ee723adaecd None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:11.708028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e96ec6-f37c-488e-b36d-8e7c0c28f4b4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:11.749907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:11.759532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11987f68-02bf-47a5-9bab-535b0cba700c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:11.760918 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11987f68-02bf-47a5-9bab-535b0cba700c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:11.763083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11987f68-02bf-47a5-9bab-535b0cba700c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:11.763083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11987f68-02bf-47a5-9bab-535b0cba700c None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:11.764041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11987f68-02bf-47a5-9bab-535b0cba700c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:11.764609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11987f68-02bf-47a5-9bab-535b0cba700c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:11.766550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11987f68-02bf-47a5-9bab-535b0cba700c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:11.775610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 365/726] 10.222.0.81 () {56 vars in 1124 bytes} [Thu Jun 16 18:38:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:11.781765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea303c-8086-44bd-aa89-ee2888d3718a None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:11.781765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea303c-8086-44bd-aa89-ee2888d3718a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:11.781765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea303c-8086-44bd-aa89-ee2888d3718a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:11.814057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e10ab-6ff4-47ec-b54d-6ee723adaecd None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:11.822395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3cea303c-8086-44bd-aa89-ee2888d3718a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:11.823628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3cea303c-8086-44bd-aa89-ee2888d3718a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:11.874879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905487f-a84e-4880-b140-d33e81024247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:11.875945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905487f-a84e-4880-b140-d33e81024247 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:11.876634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905487f-a84e-4880-b140-d33e81024247 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:11.877082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905487f-a84e-4880-b140-d33e81024247 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:11.890044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905487f-a84e-4880-b140-d33e81024247 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:11.892015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905487f-a84e-4880-b140-d33e81024247 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-6905487f-a84e-4880-b140-d33e81024247 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:11.909715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:11.910734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:11.911683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:11.911683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:11.911683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:11.911683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:11.911683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:11.911683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:11.911683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:11.911683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:11.911683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:11.911683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:11.914380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 362/727] 10.222.0.81 () {56 vars in 1117 bytes} [Thu Jun 16 18:38:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:11.920938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905487f-a84e-4880-b140-d33e81024247 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:12.021646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 366/728] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:12.042664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea303c-8086-44bd-aa89-ee2888d3718a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:12.045103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905487f-a84e-4880-b140-d33e81024247 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:12.060612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:12.090033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1657766-591a-40f9-ad99-9af0407a4bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:12.091007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1657766-591a-40f9-ad99-9af0407a4bf3 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:12.091805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1657766-591a-40f9-ad99-9af0407a4bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:12.092233 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1657766-591a-40f9-ad99-9af0407a4bf3 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:12.093594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1657766-591a-40f9-ad99-9af0407a4bf3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:12.094373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1657766-591a-40f9-ad99-9af0407a4bf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:12.096324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1657766-591a-40f9-ad99-9af0407a4bf3 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:12.119229 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 363/729] 10.222.0.81 () {56 vars in 1149 bytes} [Thu Jun 16 18:38:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:12.122478 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bfd94-7e57-459c-b628-6c1e0d8aa0b3 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:12.122922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bfd94-7e57-459c-b628-6c1e0d8aa0b3 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:12.123400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bfd94-7e57-459c-b628-6c1e0d8aa0b3 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:12.125180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 364/730] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:12.130104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bfd94-7e57-459c-b628-6c1e0d8aa0b3 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:12.135247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:12.157837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea303c-8086-44bd-aa89-ee2888d3718a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:12.194903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b7da6-40dc-4d2b-b73b-f32438dbc346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:12.195732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b7da6-40dc-4d2b-b73b-f32438dbc346 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:12.196026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b7da6-40dc-4d2b-b73b-f32438dbc346 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:12.196301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b7da6-40dc-4d2b-b73b-f32438dbc346 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:12.210969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b7da6-40dc-4d2b-b73b-f32438dbc346 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-c87b7da6-40dc-4d2b-b73b-f32438dbc346 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:12.218772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:12.219687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:12.220558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 367/731] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:12.221805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0635c5e-448d-49dd-8ccf-1e97cc758fbd None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:12.221915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0635c5e-448d-49dd-8ccf-1e97cc758fbd None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:12.222186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0635c5e-448d-49dd-8ccf-1e97cc758fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:12.242226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f0635c5e-448d-49dd-8ccf-1e97cc758fbd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:12.244415 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f0635c5e-448d-49dd-8ccf-1e97cc758fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:12.268478 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bfd94-7e57-459c-b628-6c1e0d8aa0b3 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:12.306498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def303c-be4c-4bbf-a8ed-c975ffb9f5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:12.307200 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def303c-be4c-4bbf-a8ed-c975ffb9f5b1 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:12.307541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def303c-be4c-4bbf-a8ed-c975ffb9f5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:12.307817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def303c-be4c-4bbf-a8ed-c975ffb9f5b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:12.346829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 368/732] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:12.348105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0635c5e-448d-49dd-8ccf-1e97cc758fbd None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:12.375810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:12.405586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def303c-be4c-4bbf-a8ed-c975ffb9f5b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:12.407068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def303c-be4c-4bbf-a8ed-c975ffb9f5b1 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:12.438884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:12.484247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0635c5e-448d-49dd-8ccf-1e97cc758fbd None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:12.523479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 365/733] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:38:12.524381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def303c-be4c-4bbf-a8ed-c975ffb9f5b1 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:12.530263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96829586-1e88-41d2-801c-bdc2068023e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:12.531212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96829586-1e88-41d2-801c-bdc2068023e8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:12.531697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96829586-1e88-41d2-801c-bdc2068023e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:12.532188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96829586-1e88-41d2-801c-bdc2068023e8 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:12.533657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96829586-1e88-41d2-801c-bdc2068023e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:12.534172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96829586-1e88-41d2-801c-bdc2068023e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:12.536201 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96829586-1e88-41d2-801c-bdc2068023e8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:12.548092 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 369/734] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:38:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:12.549164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96829586-1e88-41d2-801c-bdc2068023e8 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:12.609695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:12.648307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def303c-be4c-4bbf-a8ed-c975ffb9f5b1 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:12.672284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96829586-1e88-41d2-801c-bdc2068023e8 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:12.687979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf9e1f-83f7-4c54-a2f0-81519484fe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:12.688756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf9e1f-83f7-4c54-a2f0-81519484fe8b None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:12.689880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf9e1f-83f7-4c54-a2f0-81519484fe8b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:12.689880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcf9e1f-83f7-4c54-a2f0-81519484fe8b None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:12.690371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf9e1f-83f7-4c54-a2f0-81519484fe8b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:12.691835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcf9e1f-83f7-4c54-a2f0-81519484fe8b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:12.744381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:12.750782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfaadd-7f49-498a-851b-98f4a848e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:12.751628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfaadd-7f49-498a-851b-98f4a848e801 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:12.751926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfaadd-7f49-498a-851b-98f4a848e801 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:12.752205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfaadd-7f49-498a-851b-98f4a848e801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:12.763226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfaadd-7f49-498a-851b-98f4a848e801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:12.768943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfaadd-7f49-498a-851b-98f4a848e801 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-44cfaadd-7f49-498a-851b-98f4a848e801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:12.782759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:12.783664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:12.784638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:12.784638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:12.784638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:12.784638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:12.788882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 370/735] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:12.789828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfaadd-7f49-498a-851b-98f4a848e801 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:12.801282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 366/736] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:38:12 2022] POST /identity/v3/services => generated 240 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:12.804133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf9e1f-83f7-4c54-a2f0-81519484fe8b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:12.990281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfaadd-7f49-498a-851b-98f4a848e801 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:13.026344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcf9e1f-83f7-4c54-a2f0-81519484fe8b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:13.052902 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651abe3-40f5-4e75-9600-f0ccea5f66f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:13.053947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651abe3-40f5-4e75-9600-f0ccea5f66f4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:13.054300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651abe3-40f5-4e75-9600-f0ccea5f66f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:13.054645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651abe3-40f5-4e75-9600-f0ccea5f66f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:13.057971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:13.067559 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651abe3-40f5-4e75-9600-f0ccea5f66f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:13.075706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ec14e-a77e-4e03-9a13-3893ad58a9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:13.075837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651abe3-40f5-4e75-9600-f0ccea5f66f4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:13.075914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ec14e-a77e-4e03-9a13-3893ad58a9b0 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:13.075914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ec14e-a77e-4e03-9a13-3893ad58a9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:13.075914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ec14e-a77e-4e03-9a13-3893ad58a9b0 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:13.075914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ec14e-a77e-4e03-9a13-3893ad58a9b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:13.075914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ec14e-a77e-4e03-9a13-3893ad58a9b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:13.075914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ec14e-a77e-4e03-9a13-3893ad58a9b0 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:13.087399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 367/737] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-6651abe3-40f5-4e75-9600-f0ccea5f66f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:13.088351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:13.089208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:13.091442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:13.091442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:13.091442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:13.091442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:13.091442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 371/738] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:13.095525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ec14e-a77e-4e03-9a13-3893ad58a9b0 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:13.096469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651abe3-40f5-4e75-9600-f0ccea5f66f4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:13.133985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:13.221909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651abe3-40f5-4e75-9600-f0ccea5f66f4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:13.245204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ec14e-a77e-4e03-9a13-3893ad58a9b0 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:13.295610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6b33f8-ec2b-4882-a336-4e484c932071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:13.296435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b33f8-ec2b-4882-a336-4e484c932071 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:13.296719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b33f8-ec2b-4882-a336-4e484c932071 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:13.296997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6b33f8-ec2b-4882-a336-4e484c932071 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:13.312814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d643596-97b0-4de7-8882-9e47e0baa4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:13.313527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d643596-97b0-4de7-8882-9e47e0baa4bd None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:13.314095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d643596-97b0-4de7-8882-9e47e0baa4bd None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:13.314490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d643596-97b0-4de7-8882-9e47e0baa4bd None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:13.315383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d643596-97b0-4de7-8882-9e47e0baa4bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:13.315729 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d643596-97b0-4de7-8882-9e47e0baa4bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:13.317468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d643596-97b0-4de7-8882-9e47e0baa4bd None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:13.327022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b33f8-ec2b-4882-a336-4e484c932071 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:13.329835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6b33f8-ec2b-4882-a336-4e484c932071 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:13.331564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 368/739] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:13.337636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d643596-97b0-4de7-8882-9e47e0baa4bd None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-6a6b33f8-ec2b-4882-a336-4e484c932071 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:13.367350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:13.368239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:13.369505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:13.369505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:13.369505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:13.369505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:13.369505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:13.369505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:13.369505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:13.369505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:13.369505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 372/740] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:13.373083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4870878-2553-454d-953c-40349cfb0101 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:13.373083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4870878-2553-454d-953c-40349cfb0101 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:13.373083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4870878-2553-454d-953c-40349cfb0101 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:13.373528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 373/741] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:13.376105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4870878-2553-454d-953c-40349cfb0101 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:13.416597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:13.486444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d643596-97b0-4de7-8882-9e47e0baa4bd None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:13.491326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:13.504090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4870878-2553-454d-953c-40349cfb0101 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:13.525918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77e558-6434-47ef-b15d-ab1999cabfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:13.526723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77e558-6434-47ef-b15d-ab1999cabfe9 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:13.527012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77e558-6434-47ef-b15d-ab1999cabfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:13.527293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77e558-6434-47ef-b15d-ab1999cabfe9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:13.536409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77e558-6434-47ef-b15d-ab1999cabfe9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:13.537908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77e558-6434-47ef-b15d-ab1999cabfe9 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:13.548420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2138c0-faa8-4cfd-bf31-5d940a942117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:13.549394 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2138c0-faa8-4cfd-bf31-5d940a942117 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:13.549898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2138c0-faa8-4cfd-bf31-5d940a942117 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:13.550375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2138c0-faa8-4cfd-bf31-5d940a942117 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:13.551364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2138c0-faa8-4cfd-bf31-5d940a942117 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:13.551893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2138c0-faa8-4cfd-bf31-5d940a942117 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-4b77e558-6434-47ef-b15d-ab1999cabfe9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:13.553175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:13.554112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:13.555047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:13.555047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:13.555047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:13.555047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:13.555047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:13.555047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:13.555047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:13.555047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:13.555047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:38:13.555047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:13.556498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2138c0-faa8-4cfd-bf31-5d940a942117 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:13.559201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 369/742] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:13.561205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c843b1f-f11e-42a6-a716-6b037075ee7b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:13.561567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c843b1f-f11e-42a6-a716-6b037075ee7b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:13.561847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c843b1f-f11e-42a6-a716-6b037075ee7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:13.577497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 374/743] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:13.578250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2138c0-faa8-4cfd-bf31-5d940a942117 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:13.586099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-7c843b1f-f11e-42a6-a716-6b037075ee7b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:13.587368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-7c843b1f-f11e-42a6-a716-6b037075ee7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:13.662276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 370/744] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:13 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:13.663153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c843b1f-f11e-42a6-a716-6b037075ee7b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:13.704656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:13.712939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2138c0-faa8-4cfd-bf31-5d940a942117 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:13.758058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc514d8-3b7a-4b1f-bcdc-7d4718848dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:13.758659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc514d8-3b7a-4b1f-bcdc-7d4718848dc1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:13.758944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc514d8-3b7a-4b1f-bcdc-7d4718848dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:13.759220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc514d8-3b7a-4b1f-bcdc-7d4718848dc1 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:13.760854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc514d8-3b7a-4b1f-bcdc-7d4718848dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:13.761174 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc514d8-3b7a-4b1f-bcdc-7d4718848dc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:13.763371 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc514d8-3b7a-4b1f-bcdc-7d4718848dc1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:13.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c843b1f-f11e-42a6-a716-6b037075ee7b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:13.787868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 375/745] 10.222.0.81 () {56 vars in 1135 bytes} [Thu Jun 16 18:38:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:13.788930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc514d8-3b7a-4b1f-bcdc-7d4718848dc1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:13.807831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:13.826992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba199fa-b444-40ab-aaff-a46b145490ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:13.827888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba199fa-b444-40ab-aaff-a46b145490ce None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:13.828324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba199fa-b444-40ab-aaff-a46b145490ce None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:13.828750 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba199fa-b444-40ab-aaff-a46b145490ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:13.829814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba199fa-b444-40ab-aaff-a46b145490ce None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=0d78b2dc673341dba7b8cba588471c46) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:13.833949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba199fa-b444-40ab-aaff-a46b145490ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:13.833949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba199fa-b444-40ab-aaff-a46b145490ce None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:13.838473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 371/746] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:13 2022] GET /identity/v3/role_assignments?user.id=0d78b2dc673341dba7b8cba588471c46&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:13.839509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba199fa-b444-40ab-aaff-a46b145490ce None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:13.980956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc514d8-3b7a-4b1f-bcdc-7d4718848dc1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:14.026399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:14.030382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba199fa-b444-40ab-aaff-a46b145490ce None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:14.056353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a128d58-d521-4189-bbc0-270609202784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:14.057057 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128d58-d521-4189-bbc0-270609202784 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:14.057370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128d58-d521-4189-bbc0-270609202784 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:14.057660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128d58-d521-4189-bbc0-270609202784 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:14.069322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a128d58-d521-4189-bbc0-270609202784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-2a128d58-d521-4189-bbc0-270609202784 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:14.079043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:14.080221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:14.081328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 376/747] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:14.086401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a128d58-d521-4189-bbc0-270609202784 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:14.090343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b72aff-4073-4439-9afa-1117bc33ec87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:14.091621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b72aff-4073-4439-9afa-1117bc33ec87 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:14.091914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b72aff-4073-4439-9afa-1117bc33ec87 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:14.092192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b72aff-4073-4439-9afa-1117bc33ec87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:14.172254 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:14.185986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b72aff-4073-4439-9afa-1117bc33ec87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:14.187637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b72aff-4073-4439-9afa-1117bc33ec87 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:14.214466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a128d58-d521-4189-bbc0-270609202784 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:14.263705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec621f-a1d8-45e9-904d-6f8d0de110c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:14.265154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec621f-a1d8-45e9-904d-6f8d0de110c8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:14.265669 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec621f-a1d8-45e9-904d-6f8d0de110c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:14.266131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec621f-a1d8-45e9-904d-6f8d0de110c8 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:14.267685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec621f-a1d8-45e9-904d-6f8d0de110c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:14.268184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec621f-a1d8-45e9-904d-6f8d0de110c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:14.270214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec621f-a1d8-45e9-904d-6f8d0de110c8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:14.288052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 377/748] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:38:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:14.290432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f429e03-001d-4137-bc1d-d692b158f057 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:14.291176 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f429e03-001d-4137-bc1d-d692b158f057 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:14.291656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f429e03-001d-4137-bc1d-d692b158f057 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:14.294859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 378/749] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:14.296089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:14.296577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f429e03-001d-4137-bc1d-d692b158f057 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:14.300387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 372/750] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:13 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 18:38:14.306738 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dd259-1168-4421-a229-0d2c73a67497 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:14.307101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dd259-1168-4421-a229-0d2c73a67497 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:14.307460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dd259-1168-4421-a229-0d2c73a67497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:14.325268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-022dd259-1168-4421-a229-0d2c73a67497 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:14.326716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-022dd259-1168-4421-a229-0d2c73a67497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:14.402564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 373/751] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:14 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 18:38:14.404116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dd259-1168-4421-a229-0d2c73a67497 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:14.438800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f429e03-001d-4137-bc1d-d692b158f057 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:14.491537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b7c70-dbd6-46c2-a23e-43b7e9b93575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:14.492859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b7c70-dbd6-46c2-a23e-43b7e9b93575 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:14.493600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b7c70-dbd6-46c2-a23e-43b7e9b93575 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:14.494064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b7c70-dbd6-46c2-a23e-43b7e9b93575 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:14.496773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:14.509666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b7c70-dbd6-46c2-a23e-43b7e9b93575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:14.510390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dd259-1168-4421-a229-0d2c73a67497 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:14.511535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b7c70-dbd6-46c2-a23e-43b7e9b93575 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-e40b7c70-dbd6-46c2-a23e-43b7e9b93575 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:14.535413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:14.536358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:14.537279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:14.537279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:14.537279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:14.537279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:14.539871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 379/752] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:14.540786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b7c70-dbd6-46c2-a23e-43b7e9b93575 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:14.555284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:14.563076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5133880-7e04-4663-b4db-a2ad32075a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:14.564540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5133880-7e04-4663-b4db-a2ad32075a38 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:14.564978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5133880-7e04-4663-b4db-a2ad32075a38 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:14.565701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5133880-7e04-4663-b4db-a2ad32075a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:14.580285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5133880-7e04-4663-b4db-a2ad32075a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:14.581971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5133880-7e04-4663-b4db-a2ad32075a38 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-d5133880-7e04-4663-b4db-a2ad32075a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:14.606760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:14.607890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:14.609001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:14.609001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:14.609001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:14.609001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:14.611091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 374/753] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:14.612037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5133880-7e04-4663-b4db-a2ad32075a38 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:14.658587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b7c70-dbd6-46c2-a23e-43b7e9b93575 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:14.704411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786ae40-00c7-4e47-a053-f32c180074b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:14.705637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786ae40-00c7-4e47-a053-f32c180074b8 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:14.706071 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786ae40-00c7-4e47-a053-f32c180074b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:14.706497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786ae40-00c7-4e47-a053-f32c180074b8 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:14.707647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786ae40-00c7-4e47-a053-f32c180074b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:14.709965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5786ae40-00c7-4e47-a053-f32c180074b8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:14.727741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5133880-7e04-4663-b4db-a2ad32075a38 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:14.763576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048cfe5-1231-4225-80f1-58c353b277ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:14.764374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048cfe5-1231-4225-80f1-58c353b277ff None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:14.764760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048cfe5-1231-4225-80f1-58c353b277ff None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:14.765140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048cfe5-1231-4225-80f1-58c353b277ff None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:14.765990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048cfe5-1231-4225-80f1-58c353b277ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:14.766402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048cfe5-1231-4225-80f1-58c353b277ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:14.767864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048cfe5-1231-4225-80f1-58c353b277ff None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:14.784558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 375/754] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:14.784558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048cfe5-1231-4225-80f1-58c353b277ff None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:14.806361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 380/755] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:38:14 2022] POST /identity/v3/services => generated 251 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:14.806361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786ae40-00c7-4e47-a053-f32c180074b8 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:14.834479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:14.869622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:14.953179 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786ae40-00c7-4e47-a053-f32c180074b8 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:14.957646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048cfe5-1231-4225-80f1-58c353b277ff None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:14.999402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd1cd6-087b-44c8-8f1a-b44bbf4f0d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:15.001054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd1cd6-087b-44c8-8f1a-b44bbf4f0d99 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:15.001559 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd1cd6-087b-44c8-8f1a-b44bbf4f0d99 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:15.001984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd1cd6-087b-44c8-8f1a-b44bbf4f0d99 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:15.011152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd1cd6-087b-44c8-8f1a-b44bbf4f0d99 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:15.013739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd1cd6-087b-44c8-8f1a-b44bbf4f0d99 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-eacd1cd6-087b-44c8-8f1a-b44bbf4f0d99 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:15.031346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:15.032403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:15.033451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:15.033451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:15.033451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:15.033451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:15.033451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:15.033451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:15.033451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:15.033451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:15.033451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:15.033451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:15.036265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 381/756] 10.222.0.81 () {56 vars in 1093 bytes} [Thu Jun 16 18:38:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:15.040772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b4eb2-de5d-4595-9011-0b18188d0edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:15.040772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b4eb2-de5d-4595-9011-0b18188d0edc None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:15.040772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b4eb2-de5d-4595-9011-0b18188d0edc None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:15.040772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b4eb2-de5d-4595-9011-0b18188d0edc None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:15.041110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b4eb2-de5d-4595-9011-0b18188d0edc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:15.041110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b4eb2-de5d-4595-9011-0b18188d0edc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:15.041988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b4eb2-de5d-4595-9011-0b18188d0edc None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:15.042629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd1cd6-087b-44c8-8f1a-b44bbf4f0d99 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:15.071097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 376/757] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:15.077769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b4eb2-de5d-4595-9011-0b18188d0edc None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:15.192429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd1cd6-087b-44c8-8f1a-b44bbf4f0d99 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:15.194981 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:15.202929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b4eb2-de5d-4595-9011-0b18188d0edc None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:15.204718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:15.238189 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c70cb3-0d80-4200-9bf3-ff39334e28ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:15.239345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c70cb3-0d80-4200-9bf3-ff39334e28ac None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:15.239807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c70cb3-0d80-4200-9bf3-ff39334e28ac None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:15.240182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf01718-be9c-4097-910b-f8efbb506f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:15.240472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c70cb3-0d80-4200-9bf3-ff39334e28ac None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:15.241137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf01718-be9c-4097-910b-f8efbb506f07 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:15.241521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf01718-be9c-4097-910b-f8efbb506f07 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:15.241872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf01718-be9c-4097-910b-f8efbb506f07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:15.242202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c70cb3-0d80-4200-9bf3-ff39334e28ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:15.242683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c70cb3-0d80-4200-9bf3-ff39334e28ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:15.244502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c70cb3-0d80-4200-9bf3-ff39334e28ac None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:15.250153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf01718-be9c-4097-910b-f8efbb506f07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:15.251824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf01718-be9c-4097-910b-f8efbb506f07 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:15.268828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 382/758] 10.222.0.81 () {56 vars in 1133 bytes} [Thu Jun 16 18:38:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-cdf01718-be9c-4097-910b-f8efbb506f07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:15.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:15.274726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:15.275620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:15.275620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:15.275620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:15.275620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:15.275620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:15.275620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:15.275620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:15.275620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:15.275620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 377/759] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:15.277126 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c70cb3-0d80-4200-9bf3-ff39334e28ac None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:15.278559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf01718-be9c-4097-910b-f8efbb506f07 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:15.402767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c70cb3-0d80-4200-9bf3-ff39334e28ac None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:15.425133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf01718-be9c-4097-910b-f8efbb506f07 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:15.440496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51705d9-ffc2-4902-81f0-f55b4663c56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:15.441346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51705d9-ffc2-4902-81f0-f55b4663c56c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:15.441708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51705d9-ffc2-4902-81f0-f55b4663c56c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:15.442057 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51705d9-ffc2-4902-81f0-f55b4663c56c None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:15.442917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51705d9-ffc2-4902-81f0-f55b4663c56c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:15.443300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51705d9-ffc2-4902-81f0-f55b4663c56c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:15.444879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51705d9-ffc2-4902-81f0-f55b4663c56c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:15.457555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 383/760] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:15.463694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856f70c-4aee-4e4e-b513-a0bfeb00afa4 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:15.463946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856f70c-4aee-4e4e-b513-a0bfeb00afa4 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:15.464302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856f70c-4aee-4e4e-b513-a0bfeb00afa4 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:15.465988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 384/761] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:15.467291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856f70c-4aee-4e4e-b513-a0bfeb00afa4 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:15.469213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02c5d4-76dc-4615-97fc-daa546df0d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:15.470225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c5d4-76dc-4615-97fc-daa546df0d9c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:15.470958 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c5d4-76dc-4615-97fc-daa546df0d9c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:15.471393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02c5d4-76dc-4615-97fc-daa546df0d9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:15.486179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02c5d4-76dc-4615-97fc-daa546df0d9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:15.488580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02c5d4-76dc-4615-97fc-daa546df0d9c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:15.494411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-8d02c5d4-76dc-4615-97fc-daa546df0d9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:15.511575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:15.512721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:15.513907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:15.513907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:15.513907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:15.513907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:15.519868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 378/762] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:15.519868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982dfc7-bdf4-4c2d-a126-854258bbf034 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:15.519868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982dfc7-bdf4-4c2d-a126-854258bbf034 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:15.519868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982dfc7-bdf4-4c2d-a126-854258bbf034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:15.538542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-0982dfc7-bdf4-4c2d-a126-854258bbf034 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:15.539888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0982dfc7-bdf4-4c2d-a126-854258bbf034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:15.550873 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:15.605802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856f70c-4aee-4e4e-b513-a0bfeb00afa4 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:15.618615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 379/763] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:15.619678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982dfc7-bdf4-4c2d-a126-854258bbf034 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:15.664520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe66a7-28fd-4101-8211-262f3c75a33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:15.665106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe66a7-28fd-4101-8211-262f3c75a33e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:15.665426 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe66a7-28fd-4101-8211-262f3c75a33e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:15.665704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe66a7-28fd-4101-8211-262f3c75a33e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:15.666546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe66a7-28fd-4101-8211-262f3c75a33e None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=b90f65f1f04b416286142c081c92a02d) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:15.666867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe66a7-28fd-4101-8211-262f3c75a33e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:15.668294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe66a7-28fd-4101-8211-262f3c75a33e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:15.675050 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 385/764] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:15 2022] GET /identity/v3/role_assignments?user.id=b90f65f1f04b416286142c081c92a02d&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:15.683211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe66a7-28fd-4101-8211-262f3c75a33e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:15.743879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982dfc7-bdf4-4c2d-a126-854258bbf034 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:15.782742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3992d53-b292-46c1-9905-b0a6e964da5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:15.783730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3992d53-b292-46c1-9905-b0a6e964da5f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:15.784197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3992d53-b292-46c1-9905-b0a6e964da5f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:15.784680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3992d53-b292-46c1-9905-b0a6e964da5f None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:15.785603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3992d53-b292-46c1-9905-b0a6e964da5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:15.786095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3992d53-b292-46c1-9905-b0a6e964da5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:15.787708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3992d53-b292-46c1-9905-b0a6e964da5f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:15.805371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 380/765] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:15.807153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3992d53-b292-46c1-9905-b0a6e964da5f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:15.811395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:15.868621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:15.918474 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3992d53-b292-46c1-9905-b0a6e964da5f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:15.927258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe66a7-28fd-4101-8211-262f3c75a33e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:15.991087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b6246-1072-4dd8-8ca9-090e74737597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:15.991926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b6246-1072-4dd8-8ca9-090e74737597 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:15.992214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b6246-1072-4dd8-8ca9-090e74737597 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:15.992495 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b6246-1072-4dd8-8ca9-090e74737597 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:15.996703 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee91e6-91d9-422d-a84c-7db5095494f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:15.998097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee91e6-91d9-422d-a84c-7db5095494f1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:16.001473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee91e6-91d9-422d-a84c-7db5095494f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:16.001830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee91e6-91d9-422d-a84c-7db5095494f1 None admin] PATH_INFO: `/v3/services/network` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:16.002545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee91e6-91d9-422d-a84c-7db5095494f1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:16.003985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee91e6-91d9-422d-a84c-7db5095494f1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:16.007982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b6246-1072-4dd8-8ca9-090e74737597 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:16.009532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b6246-1072-4dd8-8ca9-090e74737597 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-57ee91e6-91d9-422d-a84c-7db5095494f1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:16.024710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:16.025816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:38:16.026733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:16.034641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 386/766] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:15 2022] GET /identity/v3/services/network => generated 88 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:16.040384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee91e6-91d9-422d-a84c-7db5095494f1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-c13b6246-1072-4dd8-8ca9-090e74737597 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:16.041573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:16.043511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:16.046222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:16.046222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:16.046222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:16.046222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:16.046222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:16.046222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:16.046222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:16.046222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:16.050909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 381/767] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:16.051833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b6246-1072-4dd8-8ca9-090e74737597 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:16.131110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:16.165366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee91e6-91d9-422d-a84c-7db5095494f1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:16.181789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b6246-1072-4dd8-8ca9-090e74737597 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:16.202659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661eeb5-33c9-4a90-90d5-d90ab485a2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:16.203375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661eeb5-33c9-4a90-90d5-d90ab485a2fb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:16.203658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661eeb5-33c9-4a90-90d5-d90ab485a2fb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:16.204036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661eeb5-33c9-4a90-90d5-d90ab485a2fb None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:16.204892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661eeb5-33c9-4a90-90d5-d90ab485a2fb None admin] RBAC: Adding query filter params (name=network) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:16.205203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661eeb5-33c9-4a90-90d5-d90ab485a2fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:16.206672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661eeb5-33c9-4a90-90d5-d90ab485a2fb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:16.211981 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 387/768] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:16.213356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06cf71-6a21-41e7-8795-8d07fb2a5664 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:16.213644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06cf71-6a21-41e7-8795-8d07fb2a5664 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:16.214006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06cf71-6a21-41e7-8795-8d07fb2a5664 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:16.220386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 388/769] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:16.221346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06cf71-6a21-41e7-8795-8d07fb2a5664 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:16.223166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e7d92-13b9-4e37-a737-6713b006e5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:16.224129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7d92-13b9-4e37-a737-6713b006e5de None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:16.224421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7d92-13b9-4e37-a737-6713b006e5de None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:16.224723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7d92-13b9-4e37-a737-6713b006e5de None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:16.226105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e7d92-13b9-4e37-a737-6713b006e5de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:16.226424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e7d92-13b9-4e37-a737-6713b006e5de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:16.227917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e7d92-13b9-4e37-a737-6713b006e5de None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:16.240374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 382/770] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:16.241759 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bfc09-b71f-4ebc-9f98-003214a0cf78 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:16.242052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bfc09-b71f-4ebc-9f98-003214a0cf78 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:16.242330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bfc09-b71f-4ebc-9f98-003214a0cf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:16.242914 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:16.260438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ee3bfc09-b71f-4ebc-9f98-003214a0cf78 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:16.261668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ee3bfc09-b71f-4ebc-9f98-003214a0cf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:16.337551 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 383/771] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:16.338586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bfc09-b71f-4ebc-9f98-003214a0cf78 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:16.358753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06cf71-6a21-41e7-8795-8d07fb2a5664 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:16.402661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85195a-f166-4eaf-8294-391a64729771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:16.403389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85195a-f166-4eaf-8294-391a64729771 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:16.404280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85195a-f166-4eaf-8294-391a64729771 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:16.404564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd85195a-f166-4eaf-8294-391a64729771 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:16.405291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85195a-f166-4eaf-8294-391a64729771 None admin] RBAC: Adding query filter params (type=network) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:16.405646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85195a-f166-4eaf-8294-391a64729771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:16.407021 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd85195a-f166-4eaf-8294-391a64729771 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:16.412247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 389/772] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:16.413186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85195a-f166-4eaf-8294-391a64729771 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:16.419883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:16.469085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bfc09-b71f-4ebc-9f98-003214a0cf78 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:16.490955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:16.507094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b185545-8603-484a-ad13-548e52bb439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:16.507992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b185545-8603-484a-ad13-548e52bb439d None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:16.508277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b185545-8603-484a-ad13-548e52bb439d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:16.508553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b185545-8603-484a-ad13-548e52bb439d None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/251ed026ce8046e98188066bf0bc40b7/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:16.510572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b185545-8603-484a-ad13-548e52bb439d None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=251ed026ce8046e98188066bf0bc40b7, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:16.512122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b185545-8603-484a-ad13-548e52bb439d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:16.516070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd85195a-f166-4eaf-8294-391a64729771 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:16.552411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b5186-4e79-4c48-b11f-b5d1b4abf253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:16.553208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b5186-4e79-4c48-b11f-b5d1b4abf253 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:16.553527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b5186-4e79-4c48-b11f-b5d1b4abf253 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:16.553804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b5186-4e79-4c48-b11f-b5d1b4abf253 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:16.564668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b5186-4e79-4c48-b11f-b5d1b4abf253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-cb2b5186-4e79-4c48-b11f-b5d1b4abf253 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:16.572230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:16.573140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 390/773] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:38:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:16.574403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b5186-4e79-4c48-b11f-b5d1b4abf253 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:16.592993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 384/774] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:38:16 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/251ed026ce8046e98188066bf0bc40b7/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:38:16.594091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b185545-8603-484a-ad13-548e52bb439d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:16.765763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b5186-4e79-4c48-b11f-b5d1b4abf253 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:16.801199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b185545-8603-484a-ad13-548e52bb439d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:16.815489 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ee0bc-1cbf-43f5-9986-d4765065e54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:16.816270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ee0bc-1cbf-43f5-9986-d4765065e54d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:16.816553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ee0bc-1cbf-43f5-9986-d4765065e54d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:16.816822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ee0bc-1cbf-43f5-9986-d4765065e54d None admin] PATH_INFO: `/v3/endpoints` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:16.817695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ee0bc-1cbf-43f5-9986-d4765065e54d None admin] RBAC: Adding query filter params (interface=public, service_id=7367a86e616c411a926cd729c4229892, region_id=RegionOne) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:16.818007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ee0bc-1cbf-43f5-9986-d4765065e54d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:16.819496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ee0bc-1cbf-43f5-9986-d4765065e54d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:16.823237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:16.825203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:16.829566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 391/775] 10.222.0.81 () {56 vars in 1251 bytes} [Thu Jun 16 18:38:16 2022] GET /identity/v3/endpoints?service_id=7367a86e616c411a926cd729c4229892&interface=public®ion_id=RegionOne => generated 194 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:16.835764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ee0bc-1cbf-43f5-9986-d4765065e54d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:16.853288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fcdd7-d667-4123-a86c-a7983be34fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:16.853288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fcdd7-d667-4123-a86c-a7983be34fc9 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:16.853288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fcdd7-d667-4123-a86c-a7983be34fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:16.853288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fcdd7-d667-4123-a86c-a7983be34fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:16.854182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fcdd7-d667-4123-a86c-a7983be34fc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:16.854398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fcdd7-d667-4123-a86c-a7983be34fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:16.856299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fcdd7-d667-4123-a86c-a7983be34fc9 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:16.873396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 385/776] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:38:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:16.880690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fcdd7-d667-4123-a86c-a7983be34fc9 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:16.985994 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ee0bc-1cbf-43f5-9986-d4765065e54d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:17.056267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d452fa-1618-4d4e-b538-a4200acbff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:17.056946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d452fa-1618-4d4e-b538-a4200acbff90 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:17.057233 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d452fa-1618-4d4e-b538-a4200acbff90 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:17.057533 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d452fa-1618-4d4e-b538-a4200acbff90 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:17.058223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d452fa-1618-4d4e-b538-a4200acbff90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:17.059604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d452fa-1618-4d4e-b538-a4200acbff90 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:17.066577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fcdd7-d667-4123-a86c-a7983be34fc9 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:17.081586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 392/777] 10.222.0.81 () {56 vars in 1087 bytes} [Thu Jun 16 18:38:16 2022] GET /identity/v3/services => generated 772 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:17.118763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd1218-4d25-4177-8245-fc31508dc68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:17.119541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd1218-4d25-4177-8245-fc31508dc68e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:17.119825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd1218-4d25-4177-8245-fc31508dc68e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:17.120098 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd1218-4d25-4177-8245-fc31508dc68e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:17.132808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd1218-4d25-4177-8245-fc31508dc68e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:17.134340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd1218-4d25-4177-8245-fc31508dc68e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-6dbd1218-4d25-4177-8245-fc31508dc68e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:17.154106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:17.155150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:17.157075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:17.157075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:17.157075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:17.157075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:17.157075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 386/778] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:17.161467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d452fa-1618-4d4e-b538-a4200acbff90 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:17.236676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:17.287507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d452fa-1618-4d4e-b538-a4200acbff90 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:17.329840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441e100-39e1-4330-a468-acf7e587168d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:17.330437 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441e100-39e1-4330-a468-acf7e587168d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:17.330723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441e100-39e1-4330-a468-acf7e587168d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:17.330998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441e100-39e1-4330-a468-acf7e587168d None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:17.331722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441e100-39e1-4330-a468-acf7e587168d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:17.332033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441e100-39e1-4330-a468-acf7e587168d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:17.333424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441e100-39e1-4330-a468-acf7e587168d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:17.348584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 393/779] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:17.352274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd1218-4d25-4177-8245-fc31508dc68e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:17.390843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:17.475057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd1218-4d25-4177-8245-fc31508dc68e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:17.512136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42096f00-01a6-4158-9d5a-568a14b94ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:17.513027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42096f00-01a6-4158-9d5a-568a14b94ac9 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:17.513980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42096f00-01a6-4158-9d5a-568a14b94ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:17.514258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42096f00-01a6-4158-9d5a-568a14b94ac9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:17.525628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42096f00-01a6-4158-9d5a-568a14b94ac9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:17.527099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42096f00-01a6-4158-9d5a-568a14b94ac9 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:17.530645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dc186-c338-4429-ba97-59863c2b5d0e None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:17.531094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dc186-c338-4429-ba97-59863c2b5d0e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:17.531288 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4dc186-c338-4429-ba97-59863c2b5d0e None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:17.532705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 394/780] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-42096f00-01a6-4158-9d5a-568a14b94ac9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:17.550504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:17.551456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:17.552476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:17.552476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:17.552476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:17.552476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:17.552476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:17.552476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:17.552476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:17.552476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:17.552476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:17.552476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:17.552999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922ed27-0d35-483e-83cc-6116474692a7 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:17.552999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922ed27-0d35-483e-83cc-6116474692a7 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:17.552999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922ed27-0d35-483e-83cc-6116474692a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:17.555282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 387/781] 10.222.0.81 () {56 vars in 1117 bytes} [Thu Jun 16 18:38:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:17.562272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42096f00-01a6-4158-9d5a-568a14b94ac9 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:17.583420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0922ed27-0d35-483e-83cc-6116474692a7 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:17.584537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0922ed27-0d35-483e-83cc-6116474692a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:17.667523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42096f00-01a6-4158-9d5a-568a14b94ac9 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:17.701607 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:17.711232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 395/782] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:17.713906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955493fe-557a-41a8-a0f8-30a78816af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:17.714590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955493fe-557a-41a8-a0f8-30a78816af6d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:17.714876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955493fe-557a-41a8-a0f8-30a78816af6d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:17.715156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955493fe-557a-41a8-a0f8-30a78816af6d None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:17.716059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955493fe-557a-41a8-a0f8-30a78816af6d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:17.716946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955493fe-557a-41a8-a0f8-30a78816af6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:17.718436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955493fe-557a-41a8-a0f8-30a78816af6d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:17.736674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 388/783] 10.222.0.81 () {56 vars in 1149 bytes} [Thu Jun 16 18:38:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:17.742184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922ed27-0d35-483e-83cc-6116474692a7 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:17.752381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955493fe-557a-41a8-a0f8-30a78816af6d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:17.845418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:17.923664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922ed27-0d35-483e-83cc-6116474692a7 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:17.948791 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955493fe-557a-41a8-a0f8-30a78816af6d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:17.990849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc1fa0-05cc-4f41-8eec-f2317f05a5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:17.991647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc1fa0-05cc-4f41-8eec-f2317f05a5f1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:17.991932 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc1fa0-05cc-4f41-8eec-f2317f05a5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:17.992205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc1fa0-05cc-4f41-8eec-f2317f05a5f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.012209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937ae5c-cde3-4702-acbf-f2125abab56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:18.013027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937ae5c-cde3-4702-acbf-f2125abab56b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.013896 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937ae5c-cde3-4702-acbf-f2125abab56b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.014176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937ae5c-cde3-4702-acbf-f2125abab56b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.014862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937ae5c-cde3-4702-acbf-f2125abab56b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:18.016186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937ae5c-cde3-4702-acbf-f2125abab56b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:18.019973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc1fa0-05cc-4f41-8eec-f2317f05a5f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:18.021440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc1fa0-05cc-4f41-8eec-f2317f05a5f1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-9937ae5c-cde3-4702-acbf-f2125abab56b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:18.041352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:18.042324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:18.043570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 389/784] 10.222.0.81 () {56 vars in 1129 bytes} [Thu Jun 16 18:38:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-d1cc1fa0-05cc-4f41-8eec-f2317f05a5f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:18.045267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:18.046801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:18.047952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:18.047952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:18.047952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:18.047952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:18.047952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 396/785] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:18.052303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937ae5c-cde3-4702-acbf-f2125abab56b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:18.053719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc1fa0-05cc-4f41-8eec-f2317f05a5f1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:18.078824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:18.192544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc1fa0-05cc-4f41-8eec-f2317f05a5f1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:18.204390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:18.222140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937ae5c-cde3-4702-acbf-f2125abab56b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:18.239044 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18277c-652c-4d8d-9db1-4eceb20df2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:18.239859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18277c-652c-4d8d-9db1-4eceb20df2c6 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.240424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18277c-652c-4d8d-9db1-4eceb20df2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.240718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18277c-652c-4d8d-9db1-4eceb20df2c6 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.241472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18277c-652c-4d8d-9db1-4eceb20df2c6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:18.241786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18277c-652c-4d8d-9db1-4eceb20df2c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:18.243163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18277c-652c-4d8d-9db1-4eceb20df2c6 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:18.250910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 397/786] 10.222.0.81 () {56 vars in 1124 bytes} [Thu Jun 16 18:38:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:18.252436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448f83b-723b-4d6e-856a-dcf22d77359d None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.252727 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448f83b-723b-4d6e-856a-dcf22d77359d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.253148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448f83b-723b-4d6e-856a-dcf22d77359d None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.255325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 398/787] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:18.256470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448f83b-723b-4d6e-856a-dcf22d77359d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:18.259549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35947cb3-bf98-4eac-bcef-c14f56470adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:18.260338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35947cb3-bf98-4eac-bcef-c14f56470adc None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.260721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35947cb3-bf98-4eac-bcef-c14f56470adc None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.261095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35947cb3-bf98-4eac-bcef-c14f56470adc None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.261939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35947cb3-bf98-4eac-bcef-c14f56470adc None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:18.262344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35947cb3-bf98-4eac-bcef-c14f56470adc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:18.264481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35947cb3-bf98-4eac-bcef-c14f56470adc None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:18.277017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 390/788] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:18.278471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248e4fc-d998-47f0-b313-3b19508aad60 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.278906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248e4fc-d998-47f0-b313-3b19508aad60 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.279283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248e4fc-d998-47f0-b313-3b19508aad60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.305758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-8248e4fc-d998-47f0-b313-3b19508aad60 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:18.307001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8248e4fc-d998-47f0-b313-3b19508aad60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:18.381636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:18.385852 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448f83b-723b-4d6e-856a-dcf22d77359d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:18.387301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 391/789] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:18.388219 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248e4fc-d998-47f0-b313-3b19508aad60 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:18.429458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04082f-8582-4f71-acea-288b4ad9c907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:18.430441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04082f-8582-4f71-acea-288b4ad9c907 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.430736 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04082f-8582-4f71-acea-288b4ad9c907 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.431053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04082f-8582-4f71-acea-288b4ad9c907 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.431910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04082f-8582-4f71-acea-288b4ad9c907 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:18.432227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04082f-8582-4f71-acea-288b4ad9c907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:18.433779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04082f-8582-4f71-acea-288b4ad9c907 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:18.440917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 399/790] 10.222.0.81 () {56 vars in 1124 bytes} [Thu Jun 16 18:38:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:18.442186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04082f-8582-4f71-acea-288b4ad9c907 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:18.486128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:18.494189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248e4fc-d998-47f0-b313-3b19508aad60 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:18.542243 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e2979-c718-4fa6-966a-c310f8107e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:18.543348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2979-c718-4fa6-966a-c310f8107e1e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.544079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2979-c718-4fa6-966a-c310f8107e1e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.544892 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2979-c718-4fa6-966a-c310f8107e1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.555154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e2979-c718-4fa6-966a-c310f8107e1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:18.557488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e2979-c718-4fa6-966a-c310f8107e1e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:18.563158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04082f-8582-4f71-acea-288b4ad9c907 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-307e2979-c718-4fa6-966a-c310f8107e1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:18.576066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:18.577158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:18.578365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:18.578365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:18.578365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:18.578365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:18.578365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:18.578365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:18.578365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:18.578365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:18.582166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 392/791] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:18.583169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e2979-c718-4fa6-966a-c310f8107e1e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:18.608111 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6bd65-1d55-42be-9d47-8765b584113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:18.609175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6bd65-1d55-42be-9d47-8765b584113f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.609558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6bd65-1d55-42be-9d47-8765b584113f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.609954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa6bd65-1d55-42be-9d47-8765b584113f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.625512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa6bd65-1d55-42be-9d47-8765b584113f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-0fa6bd65-1d55-42be-9d47-8765b584113f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:18.633987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:18.635839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 400/792] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:18.637934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6bd65-1d55-42be-9d47-8765b584113f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:18.660957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:18.715760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e2979-c718-4fa6-966a-c310f8107e1e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:18.746852 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:18.756812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa6bd65-1d55-42be-9d47-8765b584113f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:18.768235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe8a09-7424-4aca-82cb-56f47444e374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:18.768235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe8a09-7424-4aca-82cb-56f47444e374 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.768235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe8a09-7424-4aca-82cb-56f47444e374 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.768235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fe8a09-7424-4aca-82cb-56f47444e374 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.768804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe8a09-7424-4aca-82cb-56f47444e374 None admin] RBAC: Adding query filter params (interface=public, service_id=b6da965ddb9048beb8f7c3d027f1a69b, region_id=RegionOne) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:18.769192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe8a09-7424-4aca-82cb-56f47444e374 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:18.770969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fe8a09-7424-4aca-82cb-56f47444e374 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:18.781051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 393/793] 10.222.0.81 () {56 vars in 1251 bytes} [Thu Jun 16 18:38:18 2022] GET /identity/v3/endpoints?service_id=b6da965ddb9048beb8f7c3d027f1a69b&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:18.782138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe8a09-7424-4aca-82cb-56f47444e374 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:18.794739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45488f7f-356d-41d9-87eb-2d29a48c04b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:18.795813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45488f7f-356d-41d9-87eb-2d29a48c04b6 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.796347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45488f7f-356d-41d9-87eb-2d29a48c04b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.796822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45488f7f-356d-41d9-87eb-2d29a48c04b6 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.797978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45488f7f-356d-41d9-87eb-2d29a48c04b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:18.798594 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45488f7f-356d-41d9-87eb-2d29a48c04b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:18.800274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45488f7f-356d-41d9-87eb-2d29a48c04b6 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:18.814265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 401/794] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:18.815374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45488f7f-356d-41d9-87eb-2d29a48c04b6 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:18.896165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fe8a09-7424-4aca-82cb-56f47444e374 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:18.921978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:18.934197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25352018-592c-406e-96d2-45282ecc2aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:18.935250 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25352018-592c-406e-96d2-45282ecc2aa7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.935738 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25352018-592c-406e-96d2-45282ecc2aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.936250 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25352018-592c-406e-96d2-45282ecc2aa7 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.937196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25352018-592c-406e-96d2-45282ecc2aa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:18.937748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25352018-592c-406e-96d2-45282ecc2aa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:18.939320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25352018-592c-406e-96d2-45282ecc2aa7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:18.945280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45488f7f-356d-41d9-87eb-2d29a48c04b6 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:18.948765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 394/795] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:38:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:18.949855 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25352018-592c-406e-96d2-45282ecc2aa7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:18.989926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33e546-1922-4347-9d1b-24e69328c755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:18.991127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33e546-1922-4347-9d1b-24e69328c755 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:18.991612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33e546-1922-4347-9d1b-24e69328c755 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:18.992129 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33e546-1922-4347-9d1b-24e69328c755 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:18.993018 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33e546-1922-4347-9d1b-24e69328c755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:18.994605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33e546-1922-4347-9d1b-24e69328c755 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:19.001386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 402/796] 10.222.0.81 () {56 vars in 1087 bytes} [Thu Jun 16 18:38:18 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:19.002455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33e546-1922-4347-9d1b-24e69328c755 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:19.024723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:19.073688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25352018-592c-406e-96d2-45282ecc2aa7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:19.114409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e77a0-85fd-4270-9c44-00659cf2fb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:19.115320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e77a0-85fd-4270-9c44-00659cf2fb03 None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.115603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e77a0-85fd-4270-9c44-00659cf2fb03 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.115878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e77a0-85fd-4270-9c44-00659cf2fb03 None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/0d78b2dc673341dba7b8cba588471c46/roles/9718ec052fba437f923e8b15e08d7bed` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.135652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e77a0-85fd-4270-9c44-00659cf2fb03 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=0d78b2dc673341dba7b8cba588471c46, role_id=9718ec052fba437f923e8b15e08d7bed)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:19.137160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e77a0-85fd-4270-9c44-00659cf2fb03 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:19.180560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33e546-1922-4347-9d1b-24e69328c755 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:19.219852 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7cad9-8e71-4314-9a61-1f651aa4bd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:19.220759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7cad9-8e71-4314-9a61-1f651aa4bd91 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.221114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7cad9-8e71-4314-9a61-1f651aa4bd91 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.221522 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7cad9-8e71-4314-9a61-1f651aa4bd91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.233885 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7cad9-8e71-4314-9a61-1f651aa4bd91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:19.235351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7cad9-8e71-4314-9a61-1f651aa4bd91 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-09a7cad9-8e71-4314-9a61-1f651aa4bd91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:19.254469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:19.255389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:19.256323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:19.256323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:19.256323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:19.256323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:19.256323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 403/797] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:19.257183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9cc6d-aa51-40b6-9367-3e450b61f62d None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.258228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9cc6d-aa51-40b6-9367-3e450b61f62d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.258228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9cc6d-aa51-40b6-9367-3e450b61f62d None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.259091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 404/798] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:19.263123 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 395/799] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:38:18 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/0d78b2dc673341dba7b8cba588471c46/roles/9718ec052fba437f923e8b15e08d7bed => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:38:19.266248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9cc6d-aa51-40b6-9367-3e450b61f62d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:19.268070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860754e2-aeaa-4bf4-ace7-c676ed82e6ef None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.268387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860754e2-aeaa-4bf4-ace7-c676ed82e6ef None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.268631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860754e2-aeaa-4bf4-ace7-c676ed82e6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.288985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-860754e2-aeaa-4bf4-ace7-c676ed82e6ef None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:19.290294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-860754e2-aeaa-4bf4-ace7-c676ed82e6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:19.323855 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:19.369703 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:19.390117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 396/800] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:19.411714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860754e2-aeaa-4bf4-ace7-c676ed82e6ef None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:19.453927 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9cc6d-aa51-40b6-9367-3e450b61f62d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:19.494123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969b34e-8ddf-443b-8b97-49963138ef08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:19.494840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969b34e-8ddf-443b-8b97-49963138ef08 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.495125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969b34e-8ddf-443b-8b97-49963138ef08 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.495511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969b34e-8ddf-443b-8b97-49963138ef08 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.496169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969b34e-8ddf-443b-8b97-49963138ef08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:19.496737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969b34e-8ddf-443b-8b97-49963138ef08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:19.498737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969b34e-8ddf-443b-8b97-49963138ef08 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:19.512955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 405/801] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:19.518966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969b34e-8ddf-443b-8b97-49963138ef08 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:19.540484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860754e2-aeaa-4bf4-ace7-c676ed82e6ef None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:19.580042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c9b7d-3aee-4e56-9dc1-dc20e344fd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:19.581074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c9b7d-3aee-4e56-9dc1-dc20e344fd79 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.581630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c9b7d-3aee-4e56-9dc1-dc20e344fd79 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.582006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c9b7d-3aee-4e56-9dc1-dc20e344fd79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.592958 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c9b7d-3aee-4e56-9dc1-dc20e344fd79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-4f4c9b7d-3aee-4e56-9dc1-dc20e344fd79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:19.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:19.602129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:19.603179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:19.605111 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 397/802] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:19.613280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c9b7d-3aee-4e56-9dc1-dc20e344fd79 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:19.702238 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969b34e-8ddf-443b-8b97-49963138ef08 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:19.735421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:19.743144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f9eef-ba5a-4f7a-97ac-db4b7876cbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:19.743956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f9eef-ba5a-4f7a-97ac-db4b7876cbc1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.744237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f9eef-ba5a-4f7a-97ac-db4b7876cbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.744507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f9eef-ba5a-4f7a-97ac-db4b7876cbc1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.754154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f9eef-ba5a-4f7a-97ac-db4b7876cbc1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:19.755616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f9eef-ba5a-4f7a-97ac-db4b7876cbc1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-db9f9eef-ba5a-4f7a-97ac-db4b7876cbc1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:19.771037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:19.771991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:19.772934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:19.772934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:19.772934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:19.772934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:19.772934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:19.772934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:19.772934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:19.772934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:19.772934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:38:19.772934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:19.772934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 406/803] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:19.774076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f77aa-b152-464b-a854-e689c1f8c864 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.774076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f77aa-b152-464b-a854-e689c1f8c864 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.774325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f77aa-b152-464b-a854-e689c1f8c864 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.777334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 407/804] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:19.782232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f77aa-b152-464b-a854-e689c1f8c864 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:19.788905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:19.864810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c9b7d-3aee-4e56-9dc1-dc20e344fd79 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:19.916197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60960db4-3176-4f73-8e57-a5a386651333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:19.916868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60960db4-3176-4f73-8e57-a5a386651333 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.917148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60960db4-3176-4f73-8e57-a5a386651333 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.917441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60960db4-3176-4f73-8e57-a5a386651333 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.918179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60960db4-3176-4f73-8e57-a5a386651333 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:19.918490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60960db4-3176-4f73-8e57-a5a386651333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:19.920424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60960db4-3176-4f73-8e57-a5a386651333 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:19.923084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f77aa-b152-464b-a854-e689c1f8c864 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:19.929351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 398/805] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:38:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:19.930697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc00b65-9a78-4842-8f57-a0a0fa976321 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.930983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc00b65-9a78-4842-8f57-a0a0fa976321 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.931259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc00b65-9a78-4842-8f57-a0a0fa976321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.948353 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2bc00b65-9a78-4842-8f57-a0a0fa976321 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:19.949555 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2bc00b65-9a78-4842-8f57-a0a0fa976321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:19.968069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857d2df-a0ec-466d-a5e5-6c5678de90cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:19.969137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857d2df-a0ec-466d-a5e5-6c5678de90cd None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:19.970067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857d2df-a0ec-466d-a5e5-6c5678de90cd None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:19.970584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f857d2df-a0ec-466d-a5e5-6c5678de90cd None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:19.971963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857d2df-a0ec-466d-a5e5-6c5678de90cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:19.972661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857d2df-a0ec-466d-a5e5-6c5678de90cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:19.975244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f857d2df-a0ec-466d-a5e5-6c5678de90cd None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:19.995184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 408/806] 10.222.0.81 () {56 vars in 1135 bytes} [Thu Jun 16 18:38:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:19.996102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857d2df-a0ec-466d-a5e5-6c5678de90cd None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:20.033409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 399/807] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:20.034211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc00b65-9a78-4842-8f57-a0a0fa976321 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:20.055273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:20.068248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:20.123967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f857d2df-a0ec-466d-a5e5-6c5678de90cd None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:20.156842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc00b65-9a78-4842-8f57-a0a0fa976321 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:20.166692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f7c9f-d7a7-4882-80a4-a5e3df9d820f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:20.167588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f7c9f-d7a7-4882-80a4-a5e3df9d820f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:20.168153 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f7c9f-d7a7-4882-80a4-a5e3df9d820f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:20.168439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f7c9f-d7a7-4882-80a4-a5e3df9d820f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:20.182989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f7c9f-d7a7-4882-80a4-a5e3df9d820f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:20.184611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f7c9f-d7a7-4882-80a4-a5e3df9d820f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:20.195561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e50ea-b1a5-4cb2-a686-32d3eb698a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:20.196677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e50ea-b1a5-4cb2-a686-32d3eb698a68 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:20.197197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e50ea-b1a5-4cb2-a686-32d3eb698a68 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:20.197714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e50ea-b1a5-4cb2-a686-32d3eb698a68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-452f7c9f-d7a7-4882-80a4-a5e3df9d820f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:20.206379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:20.207294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:20.208792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:20.208792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:20.208792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:20.208792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:20.208792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 409/808] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:20.208792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f7c9f-d7a7-4882-80a4-a5e3df9d820f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:20.216990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e50ea-b1a5-4cb2-a686-32d3eb698a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:20.218526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e50ea-b1a5-4cb2-a686-32d3eb698a68 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-278e50ea-b1a5-4cb2-a686-32d3eb698a68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:20.237269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:20.238206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:20.239181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:20.239181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:20.239181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:20.239181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:20.243121 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 400/809] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:20.243953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e50ea-b1a5-4cb2-a686-32d3eb698a68 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:20.322410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:20.346242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:20.350755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f7c9f-d7a7-4882-80a4-a5e3df9d820f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:20.358774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e50ea-b1a5-4cb2-a686-32d3eb698a68 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:20.394787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7f1a2-0244-478e-9629-2c142796b5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:20.395857 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7f1a2-0244-478e-9629-2c142796b5b3 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:20.396211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7f1a2-0244-478e-9629-2c142796b5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:20.396580 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7f1a2-0244-478e-9629-2c142796b5b3 None admin] PATH_INFO: `/v3/services/network` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:20.398937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7f1a2-0244-478e-9629-2c142796b5b3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:20.400510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7f1a2-0244-478e-9629-2c142796b5b3 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:20.413270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f3a57-ef96-4f8e-8f39-208df6ffee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:20.413270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f3a57-ef96-4f8e-8f39-208df6ffee2c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:20.413270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f3a57-ef96-4f8e-8f39-208df6ffee2c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:20.413270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f3a57-ef96-4f8e-8f39-208df6ffee2c None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:20.413606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f3a57-ef96-4f8e-8f39-208df6ffee2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:20.413606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f3a57-ef96-4f8e-8f39-208df6ffee2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-78a7f1a2-0244-478e-9629-2c142796b5b3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:20.415156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:20.416349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 18:38:20.417192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:20.417985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f3a57-ef96-4f8e-8f39-208df6ffee2c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:20.418056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 410/810] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:20 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:20.418056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7f1a2-0244-478e-9629-2c142796b5b3 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:20.435022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 401/811] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:20.436203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f3a57-ef96-4f8e-8f39-208df6ffee2c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:20.553896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7f1a2-0244-478e-9629-2c142796b5b3 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:20.554316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f3a57-ef96-4f8e-8f39-208df6ffee2c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:20.587106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:20.593651 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ed301-a370-4c1f-b1f3-1f19ba7c9cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:20.601255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ed301-a370-4c1f-b1f3-1f19ba7c9cfb None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:20.601255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ed301-a370-4c1f-b1f3-1f19ba7c9cfb None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:20.601255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ed301-a370-4c1f-b1f3-1f19ba7c9cfb None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:20.601255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ed301-a370-4c1f-b1f3-1f19ba7c9cfb None admin] RBAC: Adding query filter params (name=network) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:20.601255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ed301-a370-4c1f-b1f3-1f19ba7c9cfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:20.601255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ed301-a370-4c1f-b1f3-1f19ba7c9cfb None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:20.601766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0090cf5-5c50-4b73-addd-917bed61deb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:20.601766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0090cf5-5c50-4b73-addd-917bed61deb1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:20.601766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0090cf5-5c50-4b73-addd-917bed61deb1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:20.601766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0090cf5-5c50-4b73-addd-917bed61deb1 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:20.602644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0090cf5-5c50-4b73-addd-917bed61deb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:20.602767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 402/812] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:20.603202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0090cf5-5c50-4b73-addd-917bed61deb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:20.603445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ed301-a370-4c1f-b1f3-1f19ba7c9cfb None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:20.604989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0090cf5-5c50-4b73-addd-917bed61deb1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:20.621971 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 411/813] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:20.623836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0090cf5-5c50-4b73-addd-917bed61deb1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:20.661752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:20.723817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ed301-a370-4c1f-b1f3-1f19ba7c9cfb None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:20.740583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0090cf5-5c50-4b73-addd-917bed61deb1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:20.770381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a381ea7-e868-47fd-81be-52448cd92a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:20.771171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a381ea7-e868-47fd-81be-52448cd92a08 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:20.771455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a381ea7-e868-47fd-81be-52448cd92a08 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:20.771726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a381ea7-e868-47fd-81be-52448cd92a08 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:20.782757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a381ea7-e868-47fd-81be-52448cd92a08 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:20.784198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a381ea7-e868-47fd-81be-52448cd92a08 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:20.784447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592e417-f9a0-4653-81d3-9a6f57c01c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:20.785534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592e417-f9a0-4653-81d3-9a6f57c01c3f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:20.786270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592e417-f9a0-4653-81d3-9a6f57c01c3f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:20.786730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592e417-f9a0-4653-81d3-9a6f57c01c3f None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:20.788167 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592e417-f9a0-4653-81d3-9a6f57c01c3f None admin] RBAC: Adding query filter params (type=network) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:20.788674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592e417-f9a0-4653-81d3-9a6f57c01c3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:20.790222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592e417-f9a0-4653-81d3-9a6f57c01c3f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:20.796170 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 412/814] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:20.797120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592e417-f9a0-4653-81d3-9a6f57c01c3f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-0a381ea7-e868-47fd-81be-52448cd92a08 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:20.798865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:20.799811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:20.800779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:20.800779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:20.800779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:20.800779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:20.800779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:20.800779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:20.800779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:20.800779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:20.800779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:20.800779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:20.805292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 403/815] 10.222.0.81 () {56 vars in 1093 bytes} [Thu Jun 16 18:38:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:20.805586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a381ea7-e868-47fd-81be-52448cd92a08 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:20.874687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:20.898638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:20.911175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592e417-f9a0-4653-81d3-9a6f57c01c3f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:20.940758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a381ea7-e868-47fd-81be-52448cd92a08 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:20.950782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76c765-b371-43f0-8be9-aa8bb6539c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:20.951663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76c765-b371-43f0-8be9-aa8bb6539c99 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:20.951952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76c765-b371-43f0-8be9-aa8bb6539c99 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:20.952248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76c765-b371-43f0-8be9-aa8bb6539c99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:20.961000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76c765-b371-43f0-8be9-aa8bb6539c99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:20.962953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76c765-b371-43f0-8be9-aa8bb6539c99 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:20.976999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4239e9-c98d-4850-b2f9-8c20fbeba1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:20.978030 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4239e9-c98d-4850-b2f9-8c20fbeba1c5 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:20.978514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4239e9-c98d-4850-b2f9-8c20fbeba1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:20.979009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4239e9-c98d-4850-b2f9-8c20fbeba1c5 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:20.980049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4239e9-c98d-4850-b2f9-8c20fbeba1c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:20.980558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4239e9-c98d-4850-b2f9-8c20fbeba1c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-3c76c765-b371-43f0-8be9-aa8bb6539c99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:20.980731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:20.981610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:20.983067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:20.983067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:20.983067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:20.983067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:20.983067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:20.983067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:20.983067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:20.983067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:20.983067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 413/816] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:20.983987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76c765-b371-43f0-8be9-aa8bb6539c99 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:20.985680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4239e9-c98d-4850-b2f9-8c20fbeba1c5 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:21.004522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 404/817] 10.222.0.81 () {56 vars in 1133 bytes} [Thu Jun 16 18:38:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:21.005827 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4239e9-c98d-4850-b2f9-8c20fbeba1c5 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:21.098059 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76c765-b371-43f0-8be9-aa8bb6539c99 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:21.129520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4239e9-c98d-4850-b2f9-8c20fbeba1c5 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:21.141697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ab08c-8ef8-4a4d-85ce-b43604b6ff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:21.142462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ab08c-8ef8-4a4d-85ce-b43604b6ff78 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:21.142756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ab08c-8ef8-4a4d-85ce-b43604b6ff78 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:21.143037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ab08c-8ef8-4a4d-85ce-b43604b6ff78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:21.143777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ab08c-8ef8-4a4d-85ce-b43604b6ff78 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:21.144444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:21.145209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ab08c-8ef8-4a4d-85ce-b43604b6ff78 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:21.171014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88907c85-5594-4258-b3c1-d0e31745a10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:21.171680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88907c85-5594-4258-b3c1-d0e31745a10d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:21.184656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88907c85-5594-4258-b3c1-d0e31745a10d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:21.184656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88907c85-5594-4258-b3c1-d0e31745a10d None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:21.184656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88907c85-5594-4258-b3c1-d0e31745a10d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:21.184656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88907c85-5594-4258-b3c1-d0e31745a10d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:21.184656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88907c85-5594-4258-b3c1-d0e31745a10d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:21.188380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 405/818] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:21.190828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88907c85-5594-4258-b3c1-d0e31745a10d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:21.289997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 414/819] 10.222.0.81 () {60 vars in 1144 bytes} [Thu Jun 16 18:38:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:21.291085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ab08c-8ef8-4a4d-85ce-b43604b6ff78 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:21.303697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:21.319696 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88907c85-5594-4258-b3c1-d0e31745a10d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:21.432532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ae4ce-605f-4afe-9a63-3c4bac48b71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:21.433355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ae4ce-605f-4afe-9a63-3c4bac48b71f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:21.433647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ae4ce-605f-4afe-9a63-3c4bac48b71f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:21.433926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ae4ce-605f-4afe-9a63-3c4bac48b71f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:21.492854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ab08c-8ef8-4a4d-85ce-b43604b6ff78 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:21.499900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:21.543083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ae4ce-605f-4afe-9a63-3c4bac48b71f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:21.544120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ce710-b3fc-4427-8b48-ce7c3cc2f549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:21.544547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ae4ce-605f-4afe-9a63-3c4bac48b71f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:21.545049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ce710-b3fc-4427-8b48-ce7c3cc2f549 None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:21.545362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ce710-b3fc-4427-8b48-ce7c3cc2f549 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:21.545642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ce710-b3fc-4427-8b48-ce7c3cc2f549 None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/b90f65f1f04b416286142c081c92a02d/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:21.547629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ce710-b3fc-4427-8b48-ce7c3cc2f549 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=b90f65f1f04b416286142c081c92a02d, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:21.549077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ce710-b3fc-4427-8b48-ce7c3cc2f549 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:21.626335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 415/820] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:38:21 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/b90f65f1f04b416286142c081c92a02d/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:38:21.626335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a38bd-ac9b-4ca9-8d92-41e3b97dd4d0 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:21.626335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a38bd-ac9b-4ca9-8d92-41e3b97dd4d0 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:21.626335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a38bd-ac9b-4ca9-8d92-41e3b97dd4d0 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:21.626335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 416/821] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:21.631290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067aca0-e19b-4436-8f84-31d83274c6d9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:21.631587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067aca0-e19b-4436-8f84-31d83274c6d9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:21.635363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067aca0-e19b-4436-8f84-31d83274c6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:21.652070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-b067aca0-e19b-4436-8f84-31d83274c6d9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:21.653388 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b067aca0-e19b-4436-8f84-31d83274c6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:21.670443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 406/822] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:21 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:38:21.672424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058de11-ab5e-427a-b9ed-2ea3c82067b2 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:21.672882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058de11-ab5e-427a-b9ed-2ea3c82067b2 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:21.673336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058de11-ab5e-427a-b9ed-2ea3c82067b2 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:21.674868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 407/823] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:21.676375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058de11-ab5e-427a-b9ed-2ea3c82067b2 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:21.732419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:21.740440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 417/824] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:21.741727 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73e593-6616-4e46-975c-262b66cbf9a2 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:21.742287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73e593-6616-4e46-975c-262b66cbf9a2 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:21.742287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c73e593-6616-4e46-975c-262b66cbf9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:21.769152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4c73e593-6616-4e46-975c-262b66cbf9a2 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:21.770265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:21.773393 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4c73e593-6616-4e46-975c-262b66cbf9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:21.814658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058de11-ab5e-427a-b9ed-2ea3c82067b2 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:21.849373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 418/825] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:21.850721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73e593-6616-4e46-975c-262b66cbf9a2 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:21.855558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b639a-9132-4d33-b84e-0a985169ba61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:21.856915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b639a-9132-4d33-b84e-0a985169ba61 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:21.857415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b639a-9132-4d33-b84e-0a985169ba61 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:21.857834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b639a-9132-4d33-b84e-0a985169ba61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:21.873708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b639a-9132-4d33-b84e-0a985169ba61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:21.875837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b639a-9132-4d33-b84e-0a985169ba61 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-2e2b639a-9132-4d33-b84e-0a985169ba61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:21.898585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:21.899594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:21.900496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:21.900496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:21.900496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:21.900496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:21.900496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 408/826] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:21.901266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b639a-9132-4d33-b84e-0a985169ba61 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:21.971008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c73e593-6616-4e46-975c-262b66cbf9a2 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:21.999250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:22.010114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b639a-9132-4d33-b84e-0a985169ba61 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.010940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0437b-1f74-4c2c-b3dc-4da76b34a19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.011749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0437b-1f74-4c2c-b3dc-4da76b34a19e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.012031 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0437b-1f74-4c2c-b3dc-4da76b34a19e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.012304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0437b-1f74-4c2c-b3dc-4da76b34a19e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.012951 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0437b-1f74-4c2c-b3dc-4da76b34a19e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.015013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0437b-1f74-4c2c-b3dc-4da76b34a19e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-58d0437b-1f74-4c2c-b3dc-4da76b34a19e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:22.027325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:22.028427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 18:38:22.029523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:22.031775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 419/827] 10.222.0.81 () {56 vars in 1129 bytes} [Thu Jun 16 18:38:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:22.032146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0437b-1f74-4c2c-b3dc-4da76b34a19e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.039357 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:22.054506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7be7d-a393-4308-be56-a111fe921f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.055311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7be7d-a393-4308-be56-a111fe921f47 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.055593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7be7d-a393-4308-be56-a111fe921f47 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.055864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7be7d-a393-4308-be56-a111fe921f47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.066296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7be7d-a393-4308-be56-a111fe921f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-c1b7be7d-a393-4308-be56-a111fe921f47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:22.074812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:22.075904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:38:22.077458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:22.084186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 409/828] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:38:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:22.084186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7be7d-a393-4308-be56-a111fe921f47 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.142546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0437b-1f74-4c2c-b3dc-4da76b34a19e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.178195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503a68a-2166-4376-a6f7-37b56688bd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.178968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503a68a-2166-4376-a6f7-37b56688bd80 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.179325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503a68a-2166-4376-a6f7-37b56688bd80 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.179701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503a68a-2166-4376-a6f7-37b56688bd80 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.180490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503a68a-2166-4376-a6f7-37b56688bd80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:22.180868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503a68a-2166-4376-a6f7-37b56688bd80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.182343 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503a68a-2166-4376-a6f7-37b56688bd80 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.182450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7be7d-a393-4308-be56-a111fe921f47 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.196334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 420/829] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:22.197674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503a68a-2166-4376-a6f7-37b56688bd80 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.219141 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82fe6f-73f8-4474-b37e-157116d7888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.219806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82fe6f-73f8-4474-b37e-157116d7888d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.220097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82fe6f-73f8-4474-b37e-157116d7888d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.220378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82fe6f-73f8-4474-b37e-157116d7888d None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.221085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82fe6f-73f8-4474-b37e-157116d7888d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:22.221427 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82fe6f-73f8-4474-b37e-157116d7888d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.222803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be82fe6f-73f8-4474-b37e-157116d7888d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.227802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 410/830] 10.222.0.81 () {56 vars in 1124 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:22.229136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82fe6f-73f8-4474-b37e-157116d7888d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.256536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:22.282743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:22.312726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503a68a-2166-4376-a6f7-37b56688bd80 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.338658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82fe6f-73f8-4474-b37e-157116d7888d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.348875 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728ca81-0af1-4435-8e04-9987818246b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.349324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728ca81-0af1-4435-8e04-9987818246b4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.349742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728ca81-0af1-4435-8e04-9987818246b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.350114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728ca81-0af1-4435-8e04-9987818246b4 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.351005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728ca81-0af1-4435-8e04-9987818246b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:22.351411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728ca81-0af1-4435-8e04-9987818246b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.352869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728ca81-0af1-4435-8e04-9987818246b4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.365367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 421/831] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:22.366197 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728ca81-0af1-4435-8e04-9987818246b4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.381426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955cd41-957a-46f7-9503-4d8fd341a8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.382227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955cd41-957a-46f7-9503-4d8fd341a8e2 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.382513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955cd41-957a-46f7-9503-4d8fd341a8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.382790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955cd41-957a-46f7-9503-4d8fd341a8e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.390174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955cd41-957a-46f7-9503-4d8fd341a8e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.391608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955cd41-957a-46f7-9503-4d8fd341a8e2 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-0955cd41-957a-46f7-9503-4d8fd341a8e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:22.406003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:22.407082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:22.412875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:22.412875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:22.412875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:22.412875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:22.412875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:22.412875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:22.412875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:22.412875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:22.412875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 411/832] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:22.412875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955cd41-957a-46f7-9503-4d8fd341a8e2 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.478123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728ca81-0af1-4435-8e04-9987818246b4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.488639 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:22.512982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c92c5-f8ff-4647-b331-15459a9ecadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.513751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c92c5-f8ff-4647-b331-15459a9ecadb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.514604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c92c5-f8ff-4647-b331-15459a9ecadb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.514889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c92c5-f8ff-4647-b331-15459a9ecadb None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.515651 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c92c5-f8ff-4647-b331-15459a9ecadb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:22.515963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c92c5-f8ff-4647-b331-15459a9ecadb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.516979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955cd41-957a-46f7-9503-4d8fd341a8e2 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.517360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4c92c5-f8ff-4647-b331-15459a9ecadb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.524409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 422/833] 10.222.0.81 () {56 vars in 1124 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:22.525563 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c92c5-f8ff-4647-b331-15459a9ecadb None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.527020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:22.566049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c16c9d-534f-4d90-a562-2b3e01d239c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.566841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c16c9d-534f-4d90-a562-2b3e01d239c8 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.567119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c16c9d-534f-4d90-a562-2b3e01d239c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.567388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c16c9d-534f-4d90-a562-2b3e01d239c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.578597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c16c9d-534f-4d90-a562-2b3e01d239c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.580033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c16c9d-534f-4d90-a562-2b3e01d239c8 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-74c16c9d-534f-4d90-a562-2b3e01d239c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:22.598744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:22.599874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:22.605723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:22.605723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:22.605723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:22.605723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:22.605723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 412/834] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:22.605723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c16c9d-534f-4d90-a562-2b3e01d239c8 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.640003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c92c5-f8ff-4647-b331-15459a9ecadb None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.675054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72392758-89f9-4137-8a71-c0560d7815d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.675920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72392758-89f9-4137-8a71-c0560d7815d1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.676309 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72392758-89f9-4137-8a71-c0560d7815d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.677111 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72392758-89f9-4137-8a71-c0560d7815d1 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.677606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72392758-89f9-4137-8a71-c0560d7815d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:22.678011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72392758-89f9-4137-8a71-c0560d7815d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.679624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72392758-89f9-4137-8a71-c0560d7815d1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.697804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 423/835] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:22.697804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72392758-89f9-4137-8a71-c0560d7815d1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.712306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c16c9d-534f-4d90-a562-2b3e01d239c8 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.736980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:22.745728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edabecd-e89c-4f45-a628-78d0602d7a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.746511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edabecd-e89c-4f45-a628-78d0602d7a01 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.746868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edabecd-e89c-4f45-a628-78d0602d7a01 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.747476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edabecd-e89c-4f45-a628-78d0602d7a01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.748813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edabecd-e89c-4f45-a628-78d0602d7a01 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.750293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edabecd-e89c-4f45-a628-78d0602d7a01 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.806624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72392758-89f9-4137-8a71-c0560d7815d1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:22.815984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 413/836] 10.222.0.81 () {60 vars in 1144 bytes} [Thu Jun 16 18:38:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:22.817123 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edabecd-e89c-4f45-a628-78d0602d7a01 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.840533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:22.865462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af8489-2405-47c2-983b-db399faf81ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:22.866528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af8489-2405-47c2-983b-db399faf81ce None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:22.866842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af8489-2405-47c2-983b-db399faf81ce None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:22.868001 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af8489-2405-47c2-983b-db399faf81ce None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:22.868848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af8489-2405-47c2-983b-db399faf81ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:22.869167 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af8489-2405-47c2-983b-db399faf81ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:22.870745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af8489-2405-47c2-983b-db399faf81ce None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:22.905507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 424/837] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:22.911768 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af8489-2405-47c2-983b-db399faf81ce None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:22.969518 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edabecd-e89c-4f45-a628-78d0602d7a01 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:23.021785 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8ec8f-fb28-4049-ad37-3df5bb3e275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:23.022446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8ec8f-fb28-4049-ad37-3df5bb3e275e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:23.022794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8ec8f-fb28-4049-ad37-3df5bb3e275e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:23.023081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8ec8f-fb28-4049-ad37-3df5bb3e275e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:23.023896 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8ec8f-fb28-4049-ad37-3df5bb3e275e None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=251ed026ce8046e98188066bf0bc40b7) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:23.024206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8ec8f-fb28-4049-ad37-3df5bb3e275e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:23.025704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8ec8f-fb28-4049-ad37-3df5bb3e275e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:23.032597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 414/838] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/role_assignments?user.id=251ed026ce8046e98188066bf0bc40b7&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:23.039815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:23.049023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af8489-2405-47c2-983b-db399faf81ce None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:23.091682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990935d-13ff-4fd6-8325-94539b4ae6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:23.093701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990935d-13ff-4fd6-8325-94539b4ae6a1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:23.094231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990935d-13ff-4fd6-8325-94539b4ae6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:23.094705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990935d-13ff-4fd6-8325-94539b4ae6a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:23.103628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990935d-13ff-4fd6-8325-94539b4ae6a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:23.105519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990935d-13ff-4fd6-8325-94539b4ae6a1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-6990935d-13ff-4fd6-8325-94539b4ae6a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:23.121751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:23.123069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:23.124043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:23.124043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:23.124043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:23.124043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:23.124043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:23.124043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:23.124043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:23.124043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:23.124043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:23.124043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:23.126899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 425/839] 10.222.0.81 () {56 vars in 1117 bytes} [Thu Jun 16 18:38:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:23.134454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8ec8f-fb28-4049-ad37-3df5bb3e275e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:23.221666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:23.246052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8ec8f-fb28-4049-ad37-3df5bb3e275e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:23.282684 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b7f0c-3a26-40f0-8603-2fed9eb200fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:23.283367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b7f0c-3a26-40f0-8603-2fed9eb200fd None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:23.283656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b7f0c-3a26-40f0-8603-2fed9eb200fd None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:23.283939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b7f0c-3a26-40f0-8603-2fed9eb200fd None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:23.284767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b7f0c-3a26-40f0-8603-2fed9eb200fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:23.285079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b7f0c-3a26-40f0-8603-2fed9eb200fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:23.286604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b7f0c-3a26-40f0-8603-2fed9eb200fd None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:23.305216 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 415/840] 10.222.0.81 () {56 vars in 1149 bytes} [Thu Jun 16 18:38:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:23.311429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990935d-13ff-4fd6-8325-94539b4ae6a1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:23.433967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990935d-13ff-4fd6-8325-94539b4ae6a1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:23.474423 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0b0e6-2f85-4aca-85a1-3b26d9376044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:23.475076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0b0e6-2f85-4aca-85a1-3b26d9376044 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:23.475361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0b0e6-2f85-4aca-85a1-3b26d9376044 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:23.475638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0b0e6-2f85-4aca-85a1-3b26d9376044 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:23.494864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:23.555636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0b0e6-2f85-4aca-85a1-3b26d9376044 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:23.557102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0b0e6-2f85-4aca-85a1-3b26d9376044 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:23.664812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 426/841] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:23 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:38:23.670763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b7f0c-3a26-40f0-8603-2fed9eb200fd None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:23.756717 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73d371-eeb3-4f48-b5a9-8cdb98965e88 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:23.757045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73d371-eeb3-4f48-b5a9-8cdb98965e88 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:23.757329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73d371-eeb3-4f48-b5a9-8cdb98965e88 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:23.758949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 427/842] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:23.763890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f119-ac48-4c09-8c02-a0e60e42bc46 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:23.764183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f119-ac48-4c09-8c02-a0e60e42bc46 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:23.764466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f119-ac48-4c09-8c02-a0e60e42bc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:23.781055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b7f0c-3a26-40f0-8603-2fed9eb200fd None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:23.782426 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3503f119-ac48-4c09-8c02-a0e60e42bc46 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:23.783746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3503f119-ac48-4c09-8c02-a0e60e42bc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:23.817288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffea71a-33cf-4442-be6f-e63d27412238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:23.818085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffea71a-33cf-4442-be6f-e63d27412238 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:23.818368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffea71a-33cf-4442-be6f-e63d27412238 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:23.818639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffea71a-33cf-4442-be6f-e63d27412238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:23.820536 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:23.829488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffea71a-33cf-4442-be6f-e63d27412238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:23.831040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffea71a-33cf-4442-be6f-e63d27412238 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:23.834363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-2ffea71a-33cf-4442-be6f-e63d27412238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:23.849564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:23.850708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:23.851799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:23.851799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:23.851799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:23.851799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:23.851799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 416/843] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:23.856171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffea71a-33cf-4442-be6f-e63d27412238 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:23.865405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 428/844] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:23.886205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503f119-ac48-4c09-8c02-a0e60e42bc46 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:23.984239 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffea71a-33cf-4442-be6f-e63d27412238 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:24.005695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503f119-ac48-4c09-8c02-a0e60e42bc46 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:24.020624 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcaf53-49cb-4cf7-ba34-ceda77f7d44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:24.021437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcaf53-49cb-4cf7-ba34-ceda77f7d44b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:24.021723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcaf53-49cb-4cf7-ba34-ceda77f7d44b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:24.022001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebcaf53-49cb-4cf7-ba34-ceda77f7d44b None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:24.022793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebcaf53-49cb-4cf7-ba34-ceda77f7d44b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:24.023052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebcaf53-49cb-4cf7-ba34-ceda77f7d44b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:24.024483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebcaf53-49cb-4cf7-ba34-ceda77f7d44b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:24.038604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 417/845] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:24.043516 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcaf53-49cb-4cf7-ba34-ceda77f7d44b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:24.051887 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a895fbf-0dfc-4edb-9b5c-447be3bbd036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:24.052982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a895fbf-0dfc-4edb-9b5c-447be3bbd036 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:24.053970 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a895fbf-0dfc-4edb-9b5c-447be3bbd036 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:24.054448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a895fbf-0dfc-4edb-9b5c-447be3bbd036 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:24.069165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a895fbf-0dfc-4edb-9b5c-447be3bbd036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-2a895fbf-0dfc-4edb-9b5c-447be3bbd036 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:24.079905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:24.080816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:24.081819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:24.084350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 429/846] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:24.089759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a895fbf-0dfc-4edb-9b5c-447be3bbd036 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:24.094597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:24.134824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:24.163107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebcaf53-49cb-4cf7-ba34-ceda77f7d44b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:24.204287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfab81d-a826-42fc-a762-793e181f186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:24.205089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfab81d-a826-42fc-a762-793e181f186f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:24.205406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfab81d-a826-42fc-a762-793e181f186f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:24.205632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a895fbf-0dfc-4edb-9b5c-447be3bbd036 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:24.205744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfab81d-a826-42fc-a762-793e181f186f None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:24.213323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfab81d-a826-42fc-a762-793e181f186f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:24.215230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfab81d-a826-42fc-a762-793e181f186f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-bbfab81d-a826-42fc-a762-793e181f186f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:24.233502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:24.234466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:24.235676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:24.235676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:24.235676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:24.235676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:24.235676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:24.235676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:24.235676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:24.235676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:24.235676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 418/847] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:24.240150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfab81d-a826-42fc-a762-793e181f186f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:24.248872 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97df491-fe2e-4502-8a02-1d2d2f1f223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:24.249907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97df491-fe2e-4502-8a02-1d2d2f1f223b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:24.250915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97df491-fe2e-4502-8a02-1d2d2f1f223b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:24.251454 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97df491-fe2e-4502-8a02-1d2d2f1f223b None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:24.252935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97df491-fe2e-4502-8a02-1d2d2f1f223b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:24.253521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97df491-fe2e-4502-8a02-1d2d2f1f223b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:24.256052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97df491-fe2e-4502-8a02-1d2d2f1f223b None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:24.269608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 430/848] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:38:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:24.274564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97df491-fe2e-4502-8a02-1d2d2f1f223b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:24.347157 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfab81d-a826-42fc-a762-793e181f186f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:24.361607 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:24.389221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08d6fe-7c7a-4e3a-8add-337288552d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:24.390340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08d6fe-7c7a-4e3a-8add-337288552d2a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:24.390850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08d6fe-7c7a-4e3a-8add-337288552d2a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:24.391410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08d6fe-7c7a-4e3a-8add-337288552d2a None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:24.392385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08d6fe-7c7a-4e3a-8add-337288552d2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:24.392916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08d6fe-7c7a-4e3a-8add-337288552d2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:24.394642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08d6fe-7c7a-4e3a-8add-337288552d2a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:24.405210 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97df491-fe2e-4502-8a02-1d2d2f1f223b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:24.407094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 419/849] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:24.407269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:24.414916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08d6fe-7c7a-4e3a-8add-337288552d2a None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:24.448349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07de4d9-e873-4365-8db4-6804474b2ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:24.449263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07de4d9-e873-4365-8db4-6804474b2ebb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:24.449587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07de4d9-e873-4365-8db4-6804474b2ebb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:24.449888 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07de4d9-e873-4365-8db4-6804474b2ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:24.460837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07de4d9-e873-4365-8db4-6804474b2ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:24.462581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07de4d9-e873-4365-8db4-6804474b2ebb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-a07de4d9-e873-4365-8db4-6804474b2ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:24.480367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:24.481536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:24.482442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:24.482442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:24.482442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:24.482442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:24.487419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 431/850] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:24.488226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07de4d9-e873-4365-8db4-6804474b2ebb None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:24.527622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08d6fe-7c7a-4e3a-8add-337288552d2a None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:24.565091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b97bd-919f-4492-8bf1-356b2e848ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:24.565875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b97bd-919f-4492-8bf1-356b2e848ca3 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:24.566228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b97bd-919f-4492-8bf1-356b2e848ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:24.566729 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b97bd-919f-4492-8bf1-356b2e848ca3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:24.567670 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b97bd-919f-4492-8bf1-356b2e848ca3 None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=0d78b2dc673341dba7b8cba588471c46) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:24.568079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b97bd-919f-4492-8bf1-356b2e848ca3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:24.569718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b97bd-919f-4492-8bf1-356b2e848ca3 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:24.581352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 420/851] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:24 2022] GET /identity/v3/role_assignments?user.id=0d78b2dc673341dba7b8cba588471c46&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:24.592944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07de4d9-e873-4365-8db4-6804474b2ebb None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:24.632265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca765b-6eec-4f21-a992-42eeef433ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:24.632950 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca765b-6eec-4f21-a992-42eeef433ca8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:24.633253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca765b-6eec-4f21-a992-42eeef433ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:24.633552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca765b-6eec-4f21-a992-42eeef433ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:24.634293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca765b-6eec-4f21-a992-42eeef433ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:24.634605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca765b-6eec-4f21-a992-42eeef433ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:24.636098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca765b-6eec-4f21-a992-42eeef433ca8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:24.650128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 432/852] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:24.656631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b97bd-919f-4492-8bf1-356b2e848ca3 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:24.693140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:24.770300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b97bd-919f-4492-8bf1-356b2e848ca3 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:24.804509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8268b-64fb-4bd9-8fa3-8ba8b26f53a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:24.805330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8268b-64fb-4bd9-8fa3-8ba8b26f53a6 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:24.805618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8268b-64fb-4bd9-8fa3-8ba8b26f53a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:24.805891 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8268b-64fb-4bd9-8fa3-8ba8b26f53a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:24.813708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8268b-64fb-4bd9-8fa3-8ba8b26f53a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:24.815164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8268b-64fb-4bd9-8fa3-8ba8b26f53a6 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-2ca8268b-64fb-4bd9-8fa3-8ba8b26f53a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:24.829871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:24.830837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:24.831931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:24.831931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:24.831931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:24.831931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:24.831931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:24.831931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:24.831931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:24.831931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:24.831931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 18:38:24.831931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:24.831931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 421/853] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:24.836829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca765b-6eec-4f21-a992-42eeef433ca8 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:24.858573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:24.951181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca765b-6eec-4f21-a992-42eeef433ca8 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:24.987340 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2591013-759a-4d53-a188-0fee75556de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:24.988016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2591013-759a-4d53-a188-0fee75556de3 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:24.988305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2591013-759a-4d53-a188-0fee75556de3 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:24.988580 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2591013-759a-4d53-a188-0fee75556de3 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:24.989443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2591013-759a-4d53-a188-0fee75556de3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:24.989758 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2591013-759a-4d53-a188-0fee75556de3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:24.991186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2591013-759a-4d53-a188-0fee75556de3 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:25.010326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 433/854] 10.222.0.81 () {56 vars in 1135 bytes} [Thu Jun 16 18:38:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:25.016513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8268b-64fb-4bd9-8fa3-8ba8b26f53a6 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:25.109410 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e165f7d-fb12-4cc5-bfbb-deb80cceeb34 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:25.109758 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e165f7d-fb12-4cc5-bfbb-deb80cceeb34 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:25.109976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e165f7d-fb12-4cc5-bfbb-deb80cceeb34 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:25.112050 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 434/855] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:25.118277 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc94639-8349-46ec-90e0-73026ab5792d None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:25.118564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc94639-8349-46ec-90e0-73026ab5792d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:25.118838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc94639-8349-46ec-90e0-73026ab5792d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:25.123285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:25.132883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8268b-64fb-4bd9-8fa3-8ba8b26f53a6 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:25.141016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3fc94639-8349-46ec-90e0-73026ab5792d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:25.143965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3fc94639-8349-46ec-90e0-73026ab5792d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:25.168257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77f810-e7eb-4339-a501-bb1aea3aaf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:25.168915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77f810-e7eb-4339-a501-bb1aea3aaf32 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:25.169199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77f810-e7eb-4339-a501-bb1aea3aaf32 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:25.169544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77f810-e7eb-4339-a501-bb1aea3aaf32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:25.230260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 435/856] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:25.250532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77f810-e7eb-4339-a501-bb1aea3aaf32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:25.251947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc94639-8349-46ec-90e0-73026ab5792d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:25.252072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b77f810-e7eb-4339-a501-bb1aea3aaf32 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:25.254355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:25.371615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc94639-8349-46ec-90e0-73026ab5792d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:25.374028 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:25.386339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 422/857] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:25 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:38:25.387819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0ee8d-d15a-4113-8fa5-ed69571a6a71 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:25.388123 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0ee8d-d15a-4113-8fa5-ed69571a6a71 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:25.388400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0ee8d-d15a-4113-8fa5-ed69571a6a71 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:25.397204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 423/858] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:25.397204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0ee8d-d15a-4113-8fa5-ed69571a6a71 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:25.412409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fd3d4-6c31-4aed-b0d1-d60211971d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:25.413214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fd3d4-6c31-4aed-b0d1-d60211971d78 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:25.413534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fd3d4-6c31-4aed-b0d1-d60211971d78 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:25.413808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fd3d4-6c31-4aed-b0d1-d60211971d78 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:25.414466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fd3d4-6c31-4aed-b0d1-d60211971d78 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:25.415834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fd3d4-6c31-4aed-b0d1-d60211971d78 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-658fd3d4-6c31-4aed-b0d1-d60211971d78 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:25.427965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:25.429007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:25.429909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 436/859] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:38:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:25.430702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3c272-5b9a-4d5d-b27c-968ad3bb8bab None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:25.430702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3c272-5b9a-4d5d-b27c-968ad3bb8bab None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:25.431615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3c272-5b9a-4d5d-b27c-968ad3bb8bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:25.448800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-91b3c272-5b9a-4d5d-b27c-968ad3bb8bab None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:25.450020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-91b3c272-5b9a-4d5d-b27c-968ad3bb8bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:25.505870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0ee8d-d15a-4113-8fa5-ed69571a6a71 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:25.516938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:25.525165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 437/860] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:25.534391 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3c272-5b9a-4d5d-b27c-968ad3bb8bab None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:25.550242 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fd088-e5bd-4e69-93ad-7b70a405ad85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:25.551036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fd088-e5bd-4e69-93ad-7b70a405ad85 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:25.551315 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fd088-e5bd-4e69-93ad-7b70a405ad85 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:25.551590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fd088-e5bd-4e69-93ad-7b70a405ad85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:25.562306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fd088-e5bd-4e69-93ad-7b70a405ad85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:25.563732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fd088-e5bd-4e69-93ad-7b70a405ad85 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-7d3fd088-e5bd-4e69-93ad-7b70a405ad85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:25.581973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:25.582887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:25.583862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:25.583862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:25.583862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:25.583862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:25.583862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 424/861] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:25.583862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fd088-e5bd-4e69-93ad-7b70a405ad85 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:25.628332 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:25.634128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3c272-5b9a-4d5d-b27c-968ad3bb8bab None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:25.676314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630ad75-ceae-42d7-9b65-9f2cc04b6fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:25.676980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630ad75-ceae-42d7-9b65-9f2cc04b6fb4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:25.677293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630ad75-ceae-42d7-9b65-9f2cc04b6fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:25.677561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630ad75-ceae-42d7-9b65-9f2cc04b6fb4 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:25.678381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630ad75-ceae-42d7-9b65-9f2cc04b6fb4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:25.678683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630ad75-ceae-42d7-9b65-9f2cc04b6fb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:25.680048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630ad75-ceae-42d7-9b65-9f2cc04b6fb4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:25.689352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 438/862] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:38:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:25.689352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630ad75-ceae-42d7-9b65-9f2cc04b6fb4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:25.694126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fd088-e5bd-4e69-93ad-7b70a405ad85 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:25.727847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f255412-1f2a-4863-9dc9-a5fa375f74e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:25.728658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f255412-1f2a-4863-9dc9-a5fa375f74e7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:25.728941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f255412-1f2a-4863-9dc9-a5fa375f74e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:25.729255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f255412-1f2a-4863-9dc9-a5fa375f74e7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:25.729956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f255412-1f2a-4863-9dc9-a5fa375f74e7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:25.731329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f255412-1f2a-4863-9dc9-a5fa375f74e7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-3f255412-1f2a-4863-9dc9-a5fa375f74e7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:25.744546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:25.745515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:25.746423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 425/863] 10.222.0.81 () {56 vars in 1132 bytes} [Thu Jun 16 18:38:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:25.747262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f255412-1f2a-4863-9dc9-a5fa375f74e7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:25.755894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:25.797721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630ad75-ceae-42d7-9b65-9f2cc04b6fb4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:25.833818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5382600-8653-4d5e-a4be-afec6ff99138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:25.834748 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5382600-8653-4d5e-a4be-afec6ff99138 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:25.835203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5382600-8653-4d5e-a4be-afec6ff99138 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:25.835616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5382600-8653-4d5e-a4be-afec6ff99138 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:25.836489 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5382600-8653-4d5e-a4be-afec6ff99138 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:25.836939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5382600-8653-4d5e-a4be-afec6ff99138 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:25.838896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5382600-8653-4d5e-a4be-afec6ff99138 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:25.851051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 439/864] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:25.851989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5382600-8653-4d5e-a4be-afec6ff99138 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:25.852502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f255412-1f2a-4863-9dc9-a5fa375f74e7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:25.866001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:25.894309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597e1d-d472-4594-9160-173364a9479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:25.894898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597e1d-d472-4594-9160-173364a9479c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:25.895186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597e1d-d472-4594-9160-173364a9479c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:25.895459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33597e1d-d472-4594-9160-173364a9479c None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:25.896778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33597e1d-d472-4594-9160-173364a9479c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:25.897309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33597e1d-d472-4594-9160-173364a9479c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:25.899688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33597e1d-d472-4594-9160-173364a9479c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:25.904847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 426/865] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:38:25 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:25.905539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597e1d-d472-4594-9160-173364a9479c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:25.978877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5382600-8653-4d5e-a4be-afec6ff99138 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:26.004289 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:26.020466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a9963-1d77-42ed-860f-7ba72f5df48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:26.021811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a9963-1d77-42ed-860f-7ba72f5df48f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:26.022351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a9963-1d77-42ed-860f-7ba72f5df48f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:26.022867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a9963-1d77-42ed-860f-7ba72f5df48f None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:26.023899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a9963-1d77-42ed-860f-7ba72f5df48f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:26.024421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a9963-1d77-42ed-860f-7ba72f5df48f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:26.026486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a9963-1d77-42ed-860f-7ba72f5df48f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:26.037812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 440/866] 10.222.0.81 () {56 vars in 1126 bytes} [Thu Jun 16 18:38:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:26.039525 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a9963-1d77-42ed-860f-7ba72f5df48f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:26.044733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33597e1d-d472-4594-9160-173364a9479c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:26.088980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f4dcc-c0f3-4e04-a04c-ec26c938fb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:26.089821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f4dcc-c0f3-4e04-a04c-ec26c938fb49 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:26.090111 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f4dcc-c0f3-4e04-a04c-ec26c938fb49 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:26.090386 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f4dcc-c0f3-4e04-a04c-ec26c938fb49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:26.099440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f4dcc-c0f3-4e04-a04c-ec26c938fb49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:26.101204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f4dcc-c0f3-4e04-a04c-ec26c938fb49 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-a95f4dcc-c0f3-4e04-a04c-ec26c938fb49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:26.120035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:26.120952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:26.121870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:26.121870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:26.121870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:26.121870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:26.121870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:26.121870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:26.121870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:26.121870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:26.123028 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 427/867] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:26.127740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f4dcc-c0f3-4e04-a04c-ec26c938fb49 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:26.160145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:26.172363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a9963-1d77-42ed-860f-7ba72f5df48f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:26.210173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817321d-5f0d-44d5-a805-4d4075501ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:26.211238 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817321d-5f0d-44d5-a805-4d4075501ea1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:26.211767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817321d-5f0d-44d5-a805-4d4075501ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:26.212238 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817321d-5f0d-44d5-a805-4d4075501ea1 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:26.213128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817321d-5f0d-44d5-a805-4d4075501ea1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:26.213670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817321d-5f0d-44d5-a805-4d4075501ea1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:26.215451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817321d-5f0d-44d5-a805-4d4075501ea1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:26.220855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 441/868] 10.222.0.81 () {56 vars in 1126 bytes} [Thu Jun 16 18:38:26 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:26.244782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f4dcc-c0f3-4e04-a04c-ec26c938fb49 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:26.288749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e7bc5-fa4e-44df-8407-dd686c3134d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:26.289880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e7bc5-fa4e-44df-8407-dd686c3134d7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:26.290491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e7bc5-fa4e-44df-8407-dd686c3134d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:26.291013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e7bc5-fa4e-44df-8407-dd686c3134d7 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:26.292003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e7bc5-fa4e-44df-8407-dd686c3134d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:26.292514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e7bc5-fa4e-44df-8407-dd686c3134d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:26.294294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e7bc5-fa4e-44df-8407-dd686c3134d7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:26.308635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 428/869] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:26.315479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817321d-5f0d-44d5-a805-4d4075501ea1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:26.372293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:26.430683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817321d-5f0d-44d5-a805-4d4075501ea1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:26.466144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8bfc9-7a67-4faf-9031-edebf7e2ff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:26.466809 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8bfc9-7a67-4faf-9031-edebf7e2ff39 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:26.467122 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8bfc9-7a67-4faf-9031-edebf7e2ff39 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:26.467397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8bfc9-7a67-4faf-9031-edebf7e2ff39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:26.468326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8bfc9-7a67-4faf-9031-edebf7e2ff39 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=b90f65f1f04b416286142c081c92a02d) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:26.468638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8bfc9-7a67-4faf-9031-edebf7e2ff39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:26.470128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8bfc9-7a67-4faf-9031-edebf7e2ff39 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:26.480523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 442/870] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:26 2022] GET /identity/v3/role_assignments?user.id=b90f65f1f04b416286142c081c92a02d&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:26.898440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a48e74-b14f-481a-bea2-d008833be1a9 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:26.898954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a48e74-b14f-481a-bea2-d008833be1a9 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:26.899032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a48e74-b14f-481a-bea2-d008833be1a9 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:26.900459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 429/871] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:26.906359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3b30f-b851-40a5-a7bc-3fba5549c883 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:26.907159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3b30f-b851-40a5-a7bc-3fba5549c883 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:26.907643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3b30f-b851-40a5-a7bc-3fba5549c883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:26.925998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-46a3b30f-b851-40a5-a7bc-3fba5549c883 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:26.927512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-46a3b30f-b851-40a5-a7bc-3fba5549c883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:26.995826 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 443/872] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:27.020714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a48e74-b14f-481a-bea2-d008833be1a9 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:27.119622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:27.130551 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a48e74-b14f-481a-bea2-d008833be1a9 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:27.164224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e285b39-e116-4727-be10-797a7e1d4cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:27.164992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e285b39-e116-4727-be10-797a7e1d4cbf None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:27.165297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e285b39-e116-4727-be10-797a7e1d4cbf None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:27.165572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e285b39-e116-4727-be10-797a7e1d4cbf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:27.175787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e285b39-e116-4727-be10-797a7e1d4cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-7e285b39-e116-4727-be10-797a7e1d4cbf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:27.183890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:27.184807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:27.185790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 430/873] 10.222.0.81 () {56 vars in 1120 bytes} [Thu Jun 16 18:38:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:27.191431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3b30f-b851-40a5-a7bc-3fba5549c883 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:27.235127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:27.293917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3b30f-b851-40a5-a7bc-3fba5549c883 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:27.329606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bde7da-efa5-4b3a-a992-8fa0f9a59c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:27.330267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bde7da-efa5-4b3a-a992-8fa0f9a59c2f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:27.330545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bde7da-efa5-4b3a-a992-8fa0f9a59c2f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:27.330817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bde7da-efa5-4b3a-a992-8fa0f9a59c2f None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:27.331546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bde7da-efa5-4b3a-a992-8fa0f9a59c2f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:27.331852 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bde7da-efa5-4b3a-a992-8fa0f9a59c2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:27.333208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bde7da-efa5-4b3a-a992-8fa0f9a59c2f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:27.342164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 444/874] 10.222.0.81 () {56 vars in 1115 bytes} [Thu Jun 16 18:38:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:27.349687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e285b39-e116-4727-be10-797a7e1d4cbf None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:27.471437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e285b39-e116-4727-be10-797a7e1d4cbf None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:27.509744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64afa0c6-da08-4762-b896-8d2ac4bbdf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:27.510963 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afa0c6-da08-4762-b896-8d2ac4bbdf11 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:27.511491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afa0c6-da08-4762-b896-8d2ac4bbdf11 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:27.511967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afa0c6-da08-4762-b896-8d2ac4bbdf11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:27.523523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64afa0c6-da08-4762-b896-8d2ac4bbdf11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:27.525194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64afa0c6-da08-4762-b896-8d2ac4bbdf11 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-64afa0c6-da08-4762-b896-8d2ac4bbdf11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:27.550629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:27.552440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:27.553808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:27.553808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:27.553808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:27.553808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:27.556227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 431/875] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:27.561968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bde7da-efa5-4b3a-a992-8fa0f9a59c2f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:27.665544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bde7da-efa5-4b3a-a992-8fa0f9a59c2f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:27.668331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:27.703408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2a444-c972-4da1-a226-7bae7e99ee47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:27.704124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2a444-c972-4da1-a226-7bae7e99ee47 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:27.704414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2a444-c972-4da1-a226-7bae7e99ee47 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:27.704690 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2a444-c972-4da1-a226-7bae7e99ee47 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:27.705433 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2a444-c972-4da1-a226-7bae7e99ee47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:27.705741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2a444-c972-4da1-a226-7bae7e99ee47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:27.707110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2a444-c972-4da1-a226-7bae7e99ee47 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:27.722758 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 445/876] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:27.727923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64afa0c6-da08-4762-b896-8d2ac4bbdf11 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:27.731457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:27.830403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64afa0c6-da08-4762-b896-8d2ac4bbdf11 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:27.863841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b5b45-5e6e-47d9-8480-a2aeacbb73a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:27.864660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b5b45-5e6e-47d9-8480-a2aeacbb73a4 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:27.864946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b5b45-5e6e-47d9-8480-a2aeacbb73a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:27.865225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b5b45-5e6e-47d9-8480-a2aeacbb73a4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:27.873185 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b5b45-5e6e-47d9-8480-a2aeacbb73a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:27.875209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b5b45-5e6e-47d9-8480-a2aeacbb73a4 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-3d0b5b45-5e6e-47d9-8480-a2aeacbb73a4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:27.890289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:27.892405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:27.893678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:27.893678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:27.893678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:27.893678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:27.893678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:27.893678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:27.893678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:27.893678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:27.893678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:27.893678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:27.893678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 432/877] 10.222.0.81 () {56 vars in 1117 bytes} [Thu Jun 16 18:38:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:27.897335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2a444-c972-4da1-a226-7bae7e99ee47 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:28.003389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2a444-c972-4da1-a226-7bae7e99ee47 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:28.040952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb59601-785f-4359-9fb6-9db133feed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:28.041694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59601-785f-4359-9fb6-9db133feed71 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.042159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59601-785f-4359-9fb6-9db133feed71 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.042439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59601-785f-4359-9fb6-9db133feed71 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.043291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb59601-785f-4359-9fb6-9db133feed71 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:28.043602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb59601-785f-4359-9fb6-9db133feed71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:28.045069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb59601-785f-4359-9fb6-9db133feed71 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:28.065951 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 446/878] 10.222.0.81 () {56 vars in 1149 bytes} [Thu Jun 16 18:38:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:28.071207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b5b45-5e6e-47d9-8480-a2aeacbb73a4 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:28.094211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e753e-d2d9-4776-9375-e442bf1f8ac4 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.094557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e753e-d2d9-4776-9375-e442bf1f8ac4 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.094776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e753e-d2d9-4776-9375-e442bf1f8ac4 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.096388 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 447/879] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:28.101331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da06384-ab6e-46ef-8d7b-f9406823fb50 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.101627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da06384-ab6e-46ef-8d7b-f9406823fb50 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.101912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da06384-ab6e-46ef-8d7b-f9406823fb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.119234 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6da06384-ab6e-46ef-8d7b-f9406823fb50 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:28.120623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6da06384-ab6e-46ef-8d7b-f9406823fb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:28.136814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:28.139926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:28.179035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b5b45-5e6e-47d9-8480-a2aeacbb73a4 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:28.201427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 448/880] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:28.214973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f73fd6-3c15-442f-a142-dc92c1dc52ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:28.215960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f73fd6-3c15-442f-a142-dc92c1dc52ec None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.216452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f73fd6-3c15-442f-a142-dc92c1dc52ec None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.216934 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f73fd6-3c15-442f-a142-dc92c1dc52ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.218947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da06384-ab6e-46ef-8d7b-f9406823fb50 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:28.305213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f73fd6-3c15-442f-a142-dc92c1dc52ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:28.306645 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f73fd6-3c15-442f-a142-dc92c1dc52ec None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:28.344670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da06384-ab6e-46ef-8d7b-f9406823fb50 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:28.387155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:28.394186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9167ab-faa1-4ed3-a3b4-3f98d9916d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:28.404375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9167ab-faa1-4ed3-a3b4-3f98d9916d2c None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.404375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9167ab-faa1-4ed3-a3b4-3f98d9916d2c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.404375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9167ab-faa1-4ed3-a3b4-3f98d9916d2c None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.404375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9167ab-faa1-4ed3-a3b4-3f98d9916d2c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:28.404375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9167ab-faa1-4ed3-a3b4-3f98d9916d2c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:28.429360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 433/881] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:38:28.434522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f73fd6-3c15-442f-a142-dc92c1dc52ec None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:28.488874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 449/882] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:38:28 2022] POST /identity/v3/services => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:38:28.530752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9207-5eb9-4da6-932e-19da429268d5 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.531057 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9207-5eb9-4da6-932e-19da429268d5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.531310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef9207-5eb9-4da6-932e-19da429268d5 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.533071 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 450/883] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:28.538526 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c0e05-a3bd-493e-b7b8-1d861882927f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.538812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c0e05-a3bd-493e-b7b8-1d861882927f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.539092 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9c0e05-a3bd-493e-b7b8-1d861882927f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.558573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f73fd6-3c15-442f-a142-dc92c1dc52ec None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:28.560549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ac9c0e05-a3bd-493e-b7b8-1d861882927f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:28.562640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ac9c0e05-a3bd-493e-b7b8-1d861882927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:28.571729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:28.607635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b85490-05e5-47fb-a4aa-eba279d6a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:28.608470 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b85490-05e5-47fb-a4aa-eba279d6a79f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.608767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b85490-05e5-47fb-a4aa-eba279d6a79f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.609014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b85490-05e5-47fb-a4aa-eba279d6a79f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.622531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b85490-05e5-47fb-a4aa-eba279d6a79f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:28.626823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b85490-05e5-47fb-a4aa-eba279d6a79f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-06b85490-05e5-47fb-a4aa-eba279d6a79f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:28.644719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:28.645712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:28.646604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:28.646604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:28.646604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:28.646604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:28.651956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 434/884] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:28.655465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b85490-05e5-47fb-a4aa-eba279d6a79f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:28.666843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:28.669033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 451/885] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:28 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:28.694548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c0e05-a3bd-493e-b7b8-1d861882927f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:28.789720 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b85490-05e5-47fb-a4aa-eba279d6a79f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:28.810832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9c0e05-a3bd-493e-b7b8-1d861882927f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:28.824155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84426a1e-e391-46f1-8b54-1d24da88c36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:28.824850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426a1e-e391-46f1-8b54-1d24da88c36c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.825139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426a1e-e391-46f1-8b54-1d24da88c36c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.825453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426a1e-e391-46f1-8b54-1d24da88c36c None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.826160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84426a1e-e391-46f1-8b54-1d24da88c36c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:28.826492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84426a1e-e391-46f1-8b54-1d24da88c36c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:28.827907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84426a1e-e391-46f1-8b54-1d24da88c36c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:28.843680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 435/886] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:28.843680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f56a50-5969-4b89-8261-e7ffb80e9a3c None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.843680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f56a50-5969-4b89-8261-e7ffb80e9a3c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.843680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f56a50-5969-4b89-8261-e7ffb80e9a3c None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.843680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 436/887] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:28.846621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f56a50-5969-4b89-8261-e7ffb80e9a3c None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:28.854769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b903de-5a89-4d1c-8aad-56f3954f3c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:28.855432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b903de-5a89-4d1c-8aad-56f3954f3c0f None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.855719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b903de-5a89-4d1c-8aad-56f3954f3c0f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.855993 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b903de-5a89-4d1c-8aad-56f3954f3c0f None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.856708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b903de-5a89-4d1c-8aad-56f3954f3c0f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:28.858640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b903de-5a89-4d1c-8aad-56f3954f3c0f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:28.915747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:28.947538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f56a50-5969-4b89-8261-e7ffb80e9a3c None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:28.961266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 452/888] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:38:28 2022] POST /identity/v3/services => generated 269 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:38:28.961266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327748-c423-47da-a524-0f052e4cf72a None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:28.961266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327748-c423-47da-a524-0f052e4cf72a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:28.961266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327748-c423-47da-a524-0f052e4cf72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:28.961920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:28.973557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-98327748-c423-47da-a524-0f052e4cf72a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:28.974967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-98327748-c423-47da-a524-0f052e4cf72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:29.011520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c6a80-e38c-4699-9b72-84a4e00d7921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:29.013226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c6a80-e38c-4699-9b72-84a4e00d7921 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:29.013789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c6a80-e38c-4699-9b72-84a4e00d7921 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:29.014421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c6a80-e38c-4699-9b72-84a4e00d7921 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:29.024779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c6a80-e38c-4699-9b72-84a4e00d7921 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:29.027078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c6a80-e38c-4699-9b72-84a4e00d7921 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-d18c6a80-e38c-4699-9b72-84a4e00d7921 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:29.048329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:29.049347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:29.050279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:29.050279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:29.050279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:29.050279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:29.050279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:29.050279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:29.050279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:29.050279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:29.052739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 437/889] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:29.057083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c6a80-e38c-4699-9b72-84a4e00d7921 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:29.059013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 453/890] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:29.082791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98327748-c423-47da-a524-0f052e4cf72a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:29.186900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c6a80-e38c-4699-9b72-84a4e00d7921 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:29.214613 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:29.216398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98327748-c423-47da-a524-0f052e4cf72a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:29.224036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647485c-efab-4095-9b07-5729dc00a5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:29.224701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647485c-efab-4095-9b07-5729dc00a5d6 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:29.224985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647485c-efab-4095-9b07-5729dc00a5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:29.225327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647485c-efab-4095-9b07-5729dc00a5d6 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:29.226062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647485c-efab-4095-9b07-5729dc00a5d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:29.226371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647485c-efab-4095-9b07-5729dc00a5d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:29.227801 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647485c-efab-4095-9b07-5729dc00a5d6 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:29.240476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 438/891] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:29.245897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647485c-efab-4095-9b07-5729dc00a5d6 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:29.252821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:29.258863 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbcc9c-9254-4799-b55f-cc65502bcf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:29.260158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbcc9c-9254-4799-b55f-cc65502bcf8f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:29.260521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbcc9c-9254-4799-b55f-cc65502bcf8f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:29.260872 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbcc9c-9254-4799-b55f-cc65502bcf8f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:29.262243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbcc9c-9254-4799-b55f-cc65502bcf8f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:29.263993 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbcc9c-9254-4799-b55f-cc65502bcf8f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-d5dbcc9c-9254-4799-b55f-cc65502bcf8f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:29.279564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:29.281136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:29.282091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 454/892] 10.222.0.81 () {56 vars in 1126 bytes} [Thu Jun 16 18:38:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:29.289422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbcc9c-9254-4799-b55f-cc65502bcf8f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:29.363682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647485c-efab-4095-9b07-5729dc00a5d6 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:29.399954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d94a79-e39e-4c2c-97a2-1102d901286a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:29.400993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d94a79-e39e-4c2c-97a2-1102d901286a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:29.401533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d94a79-e39e-4c2c-97a2-1102d901286a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:29.402016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d94a79-e39e-4c2c-97a2-1102d901286a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:29.403044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d94a79-e39e-4c2c-97a2-1102d901286a None admin] RBAC: Adding query filter params (role.id=df2b07cfe31149c3990f1f66fdfaed9d, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=0d78b2dc673341dba7b8cba588471c46) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:29.403565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d94a79-e39e-4c2c-97a2-1102d901286a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:29.405778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d94a79-e39e-4c2c-97a2-1102d901286a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:29.412523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 439/893] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:29 2022] GET /identity/v3/role_assignments?user.id=0d78b2dc673341dba7b8cba588471c46&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=df2b07cfe31149c3990f1f66fdfaed9d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:29.426616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbcc9c-9254-4799-b55f-cc65502bcf8f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:29.466833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059bf21-cafc-46eb-8ae3-356ddd319dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:29.467568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059bf21-cafc-46eb-8ae3-356ddd319dbd None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:29.467854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059bf21-cafc-46eb-8ae3-356ddd319dbd None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:29.468205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059bf21-cafc-46eb-8ae3-356ddd319dbd None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:29.468926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059bf21-cafc-46eb-8ae3-356ddd319dbd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:29.469320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059bf21-cafc-46eb-8ae3-356ddd319dbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:29.470753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059bf21-cafc-46eb-8ae3-356ddd319dbd None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:29.476916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 455/894] 10.222.0.81 () {56 vars in 1122 bytes} [Thu Jun 16 18:38:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:29.482989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d94a79-e39e-4c2c-97a2-1102d901286a None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:29.544290 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:29.600627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d94a79-e39e-4c2c-97a2-1102d901286a None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:29.639149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6122e66-8187-42a3-b0d1-00b49c316c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:29.640291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6122e66-8187-42a3-b0d1-00b49c316c49 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:29.640574 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6122e66-8187-42a3-b0d1-00b49c316c49 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:29.640850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6122e66-8187-42a3-b0d1-00b49c316c49 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:29.641593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6122e66-8187-42a3-b0d1-00b49c316c49 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:29.641901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6122e66-8187-42a3-b0d1-00b49c316c49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:29.643271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6122e66-8187-42a3-b0d1-00b49c316c49 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:29.648350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 440/895] 10.222.0.81 () {56 vars in 1122 bytes} [Thu Jun 16 18:38:29 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:30.746145 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad124a2-7eae-43d9-9e08-1d178242c706 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:30.746665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad124a2-7eae-43d9-9e08-1d178242c706 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:30.746665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad124a2-7eae-43d9-9e08-1d178242c706 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:30.748739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 456/896] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:30.754961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84b2cf-6922-4f80-85c8-f7baaace9b0d None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:30.755466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84b2cf-6922-4f80-85c8-f7baaace9b0d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:30.755542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84b2cf-6922-4f80-85c8-f7baaace9b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:30.772674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-9a84b2cf-6922-4f80-85c8-f7baaace9b0d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:30.773887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-9a84b2cf-6922-4f80-85c8-f7baaace9b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:30.853348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 441/897] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:30.869577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad124a2-7eae-43d9-9e08-1d178242c706 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:30.932266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:30.975120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad124a2-7eae-43d9-9e08-1d178242c706 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:31.014292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7602277-7f7f-49bc-8fa1-1d56acfabfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:31.015143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7602277-7f7f-49bc-8fa1-1d56acfabfd2 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.015429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7602277-7f7f-49bc-8fa1-1d56acfabfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.015705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7602277-7f7f-49bc-8fa1-1d56acfabfd2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.016368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7602277-7f7f-49bc-8fa1-1d56acfabfd2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:31.017860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7602277-7f7f-49bc-8fa1-1d56acfabfd2 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-b7602277-7f7f-49bc-8fa1-1d56acfabfd2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:31.029986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:31.031512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:31.032380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 457/898] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:38:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:31.037814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84b2cf-6922-4f80-85c8-f7baaace9b0d None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:31.085056 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:31.142259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84b2cf-6922-4f80-85c8-f7baaace9b0d None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:31.181745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd04552-7b53-4c59-be10-14bcbe4e7308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:31.182664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd04552-7b53-4c59-be10-14bcbe4e7308 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.183096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd04552-7b53-4c59-be10-14bcbe4e7308 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.183555 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd04552-7b53-4c59-be10-14bcbe4e7308 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.184401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd04552-7b53-4c59-be10-14bcbe4e7308 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:31.184857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd04552-7b53-4c59-be10-14bcbe4e7308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:31.186429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd04552-7b53-4c59-be10-14bcbe4e7308 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:31.196171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 442/899] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:38:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:31.197736 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7602277-7f7f-49bc-8fa1-1d56acfabfd2 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:31.209890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f73c54-aa07-43af-9807-50b4749f9d2a None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.210382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f73c54-aa07-43af-9807-50b4749f9d2a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.210801 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f73c54-aa07-43af-9807-50b4749f9d2a None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.212715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 443/900] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:31.217144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55accc46-99ea-4fc2-94f2-073aa457a6f8 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.217462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55accc46-99ea-4fc2-94f2-073aa457a6f8 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.218310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55accc46-99ea-4fc2-94f2-073aa457a6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.235983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-55accc46-99ea-4fc2-94f2-073aa457a6f8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:31.237177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-55accc46-99ea-4fc2-94f2-073aa457a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:31.313268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 444/901] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:31.317270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7602277-7f7f-49bc-8fa1-1d56acfabfd2 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:31.333551 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55accc46-99ea-4fc2-94f2-073aa457a6f8 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:31.349509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:31.371300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468f959-8af8-49d9-8147-82b856ea3c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:31.372346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f959-8af8-49d9-8147-82b856ea3c5c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.373079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f959-8af8-49d9-8147-82b856ea3c5c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.374248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468f959-8af8-49d9-8147-82b856ea3c5c None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.375422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468f959-8af8-49d9-8147-82b856ea3c5c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:31.375930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468f959-8af8-49d9-8147-82b856ea3c5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:31.379328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468f959-8af8-49d9-8147-82b856ea3c5c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:31.384851 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:31.386180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 458/902] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:38:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:31.392938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468f959-8af8-49d9-8147-82b856ea3c5c None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:31.464740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55accc46-99ea-4fc2-94f2-073aa457a6f8 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:31.502393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6e255-08aa-43a2-86ca-8ed020121ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:31.503581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6e255-08aa-43a2-86ca-8ed020121ce1 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.504205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6e255-08aa-43a2-86ca-8ed020121ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.504674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6e255-08aa-43a2-86ca-8ed020121ce1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.505619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6e255-08aa-43a2-86ca-8ed020121ce1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:31.507199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6e255-08aa-43a2-86ca-8ed020121ce1 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:31.507764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468f959-8af8-49d9-8147-82b856ea3c5c None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-cab6e255-08aa-43a2-86ca-8ed020121ce1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:31.521174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:31.522174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:38:31.522996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:31.530811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 445/903] 10.222.0.81 () {56 vars in 1132 bytes} [Thu Jun 16 18:38:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:31.532538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6e255-08aa-43a2-86ca-8ed020121ce1 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:31.545707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f236fc2-5b08-4724-bb7e-82c1f9318df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:31.546428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f236fc2-5b08-4724-bb7e-82c1f9318df7 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.546712 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f236fc2-5b08-4724-bb7e-82c1f9318df7 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.546985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f236fc2-5b08-4724-bb7e-82c1f9318df7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.547926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f236fc2-5b08-4724-bb7e-82c1f9318df7 None admin] RBAC: Adding query filter params (interface=public, service_id=1ca0673b91a846dcbcc4a1b6d4f67d64, region_id=RegionOne) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:31.548112 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f236fc2-5b08-4724-bb7e-82c1f9318df7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:31.549552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f236fc2-5b08-4724-bb7e-82c1f9318df7 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:31.560148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 459/904] 10.222.0.81 () {56 vars in 1251 bytes} [Thu Jun 16 18:38:31 2022] GET /identity/v3/endpoints?service_id=1ca0673b91a846dcbcc4a1b6d4f67d64&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:31.562815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f236fc2-5b08-4724-bb7e-82c1f9318df7 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:31.617971 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:31.645775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6e255-08aa-43a2-86ca-8ed020121ce1 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:31.647292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:31.669508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f236fc2-5b08-4724-bb7e-82c1f9318df7 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:31.679259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5b13e-0bc5-4a2e-b482-f102ed2dc607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:31.679916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5b13e-0bc5-4a2e-b482-f102ed2dc607 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.680197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5b13e-0bc5-4a2e-b482-f102ed2dc607 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.680468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5b13e-0bc5-4a2e-b482-f102ed2dc607 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.681151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5b13e-0bc5-4a2e-b482-f102ed2dc607 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:31.681483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5b13e-0bc5-4a2e-b482-f102ed2dc607 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:31.682853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5b13e-0bc5-4a2e-b482-f102ed2dc607 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:31.687910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 446/905] 10.222.0.81 () {56 vars in 1126 bytes} [Thu Jun 16 18:38:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:31.689120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92fa4b-644e-444b-82c9-4e92e2f153d8 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.689434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92fa4b-644e-444b-82c9-4e92e2f153d8 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.689708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92fa4b-644e-444b-82c9-4e92e2f153d8 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.693305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 447/906] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:31.693305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92fa4b-644e-444b-82c9-4e92e2f153d8 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:31.719650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ed01f-3f36-41c9-a11d-6b371a6d07eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:31.720325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ed01f-3f36-41c9-a11d-6b371a6d07eb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.720668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ed01f-3f36-41c9-a11d-6b371a6d07eb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.720954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ed01f-3f36-41c9-a11d-6b371a6d07eb None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.721828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ed01f-3f36-41c9-a11d-6b371a6d07eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:31.723936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ed01f-3f36-41c9-a11d-6b371a6d07eb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:31.737652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 460/907] 10.222.0.81 () {56 vars in 1087 bytes} [Thu Jun 16 18:38:31 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:38:31.740123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e6ef4-40ff-421a-9716-d963d6c7e86d None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.740123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e6ef4-40ff-421a-9716-d963d6c7e86d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.740123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e6ef4-40ff-421a-9716-d963d6c7e86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.763205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-fb5e6ef4-40ff-421a-9716-d963d6c7e86d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:31.764591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-fb5e6ef4-40ff-421a-9716-d963d6c7e86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:31.825850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92fa4b-644e-444b-82c9-4e92e2f153d8 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:31.863156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef979d6e-3293-434f-b8df-a46f67a5c288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:31.863833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef979d6e-3293-434f-b8df-a46f67a5c288 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.864114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef979d6e-3293-434f-b8df-a46f67a5c288 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.864385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef979d6e-3293-434f-b8df-a46f67a5c288 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.865074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef979d6e-3293-434f-b8df-a46f67a5c288 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:31.865429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef979d6e-3293-434f-b8df-a46f67a5c288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:31.866812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef979d6e-3293-434f-b8df-a46f67a5c288 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:31.872456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 448/908] 10.222.0.81 () {56 vars in 1126 bytes} [Thu Jun 16 18:38:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:31.874091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1feed-f9b4-4f4b-a7d2-edaf2a353db6 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.874481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1feed-f9b4-4f4b-a7d2-edaf2a353db6 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.874678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1feed-f9b4-4f4b-a7d2-edaf2a353db6 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.874884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 461/909] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:31.875979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 449/910] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:31.885348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e6ef4-40ff-421a-9716-d963d6c7e86d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:31.885539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d7961-5afc-4a2f-9b42-505182408bde None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:31.885539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d7961-5afc-4a2f-9b42-505182408bde None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:31.885539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d7961-5afc-4a2f-9b42-505182408bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:31.903251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d70d7961-5afc-4a2f-9b42-505182408bde None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:31.904447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d70d7961-5afc-4a2f-9b42-505182408bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:31.930244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:31.934127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:31.977427 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 450/911] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:31.977427 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d7961-5afc-4a2f-9b42-505182408bde None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:32.006156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e6ef4-40ff-421a-9716-d963d6c7e86d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:32.042325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bd1f0-3961-40ef-a24d-841e239a59e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:32.043100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bd1f0-3961-40ef-a24d-841e239a59e6 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:32.043389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bd1f0-3961-40ef-a24d-841e239a59e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:32.043664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bd1f0-3961-40ef-a24d-841e239a59e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:32.044448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bd1f0-3961-40ef-a24d-841e239a59e6 None admin] RBAC: Adding query filter params (interface=public, service_id=f9b95e03974a4a66a4022ceb06809036, region_id=RegionOne) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:32.044760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bd1f0-3961-40ef-a24d-841e239a59e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:32.047054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bd1f0-3961-40ef-a24d-841e239a59e6 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:32.056699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 462/912] 10.222.0.81 () {56 vars in 1251 bytes} [Thu Jun 16 18:38:31 2022] GET /identity/v3/endpoints?service_id=f9b95e03974a4a66a4022ceb06809036&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:32.056699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bd1f0-3961-40ef-a24d-841e239a59e6 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:32.101925 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d7961-5afc-4a2f-9b42-505182408bde None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:32.140252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c2871-074a-4d53-ac52-180ed9ab95f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:32.140982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c2871-074a-4d53-ac52-180ed9ab95f1 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:32.141288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c2871-074a-4d53-ac52-180ed9ab95f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:32.141562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c2871-074a-4d53-ac52-180ed9ab95f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:32.152688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c2871-074a-4d53-ac52-180ed9ab95f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:32.161155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bd1f0-3961-40ef-a24d-841e239a59e6 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-276c2871-074a-4d53-ac52-180ed9ab95f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:32.161579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:32.162515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:32.163399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 451/913] 10.222.0.81 () {56 vars in 1120 bytes} [Thu Jun 16 18:38:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:32.164106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c2871-074a-4d53-ac52-180ed9ab95f1 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:32.174217 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:32.193214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:32.197015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ca3b9-873c-46fc-a796-7f77fca2fb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:32.197848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ca3b9-873c-46fc-a796-7f77fca2fb68 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:32.198072 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ca3b9-873c-46fc-a796-7f77fca2fb68 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:32.198444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ca3b9-873c-46fc-a796-7f77fca2fb68 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:32.199320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ca3b9-873c-46fc-a796-7f77fca2fb68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:32.200550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ca3b9-873c-46fc-a796-7f77fca2fb68 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:32.275453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c2871-074a-4d53-ac52-180ed9ab95f1 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:32.308542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 463/914] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:38:32 2022] POST /identity/v3/services => generated 241 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:32.309548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ca3b9-873c-46fc-a796-7f77fca2fb68 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:32.327268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd119842-7991-4137-b574-4998f05689e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:32.332752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd119842-7991-4137-b574-4998f05689e2 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:32.332752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd119842-7991-4137-b574-4998f05689e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:32.332752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd119842-7991-4137-b574-4998f05689e2 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:32.332752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd119842-7991-4137-b574-4998f05689e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:32.332752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd119842-7991-4137-b574-4998f05689e2 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:32.339422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 452/915] 10.222.0.81 () {56 vars in 1087 bytes} [Thu Jun 16 18:38:32 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:38:32.465363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ca3b9-873c-46fc-a796-7f77fca2fb68 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:32.508302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d4be4-9876-4095-b7f2-57e0680b242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:32.508994 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d4be4-9876-4095-b7f2-57e0680b242e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:32.509311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d4be4-9876-4095-b7f2-57e0680b242e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:32.509592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d4be4-9876-4095-b7f2-57e0680b242e None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:32.510349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d4be4-9876-4095-b7f2-57e0680b242e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:32.510660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d4be4-9876-4095-b7f2-57e0680b242e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:32.512039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d4be4-9876-4095-b7f2-57e0680b242e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:32.521429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 464/916] 10.222.0.81 () {56 vars in 1115 bytes} [Thu Jun 16 18:38:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:32.526833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd119842-7991-4137-b574-4998f05689e2 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:32.617201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:32.636837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd119842-7991-4137-b574-4998f05689e2 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:32.676475 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dc0a2-d2d0-4c55-b484-427f02eb2fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:32.677301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dc0a2-d2d0-4c55-b484-427f02eb2fa7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:32.677593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dc0a2-d2d0-4c55-b484-427f02eb2fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:32.677871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9dc0a2-d2d0-4c55-b484-427f02eb2fa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:32.688539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9dc0a2-d2d0-4c55-b484-427f02eb2fa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:32.690115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9dc0a2-d2d0-4c55-b484-427f02eb2fa7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-2d9dc0a2-d2d0-4c55-b484-427f02eb2fa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:32.708808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:32.709845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:32.710811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:32.710811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:32.710811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:32.710811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:32.710811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 453/917] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:32.717131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d4be4-9876-4095-b7f2-57e0680b242e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:32.749794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:32.823877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d4be4-9876-4095-b7f2-57e0680b242e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:32.862606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff9e71-2a14-42db-942a-23e6dc2109ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:32.863275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff9e71-2a14-42db-942a-23e6dc2109ad None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:32.863562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff9e71-2a14-42db-942a-23e6dc2109ad None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:32.863842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff9e71-2a14-42db-942a-23e6dc2109ad None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:32.864568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff9e71-2a14-42db-942a-23e6dc2109ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:32.864880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff9e71-2a14-42db-942a-23e6dc2109ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:32.866285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff9e71-2a14-42db-942a-23e6dc2109ad None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:32.878923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 465/918] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:32.884525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dc0a2-d2d0-4c55-b484-427f02eb2fa7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:32.992367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9dc0a2-d2d0-4c55-b484-427f02eb2fa7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:33.023394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:33.028954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b4bf5-19c7-42b0-87c5-e5cb31c8fd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:33.029797 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b4bf5-19c7-42b0-87c5-e5cb31c8fd46 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:33.030086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b4bf5-19c7-42b0-87c5-e5cb31c8fd46 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:33.030406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61b4bf5-19c7-42b0-87c5-e5cb31c8fd46 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:33.041812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b4bf5-19c7-42b0-87c5-e5cb31c8fd46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:33.051027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61b4bf5-19c7-42b0-87c5-e5cb31c8fd46 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-f61b4bf5-19c7-42b0-87c5-e5cb31c8fd46 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:33.058792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:33.059768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:33.060805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:33.060805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:33.060805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:33.060805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:33.060805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:33.060805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:33.060805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:33.060805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:33.060805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:33.060805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:33.060805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 454/919] 10.222.0.81 () {56 vars in 1117 bytes} [Thu Jun 16 18:38:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:33.068396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff9e71-2a14-42db-942a-23e6dc2109ad None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:33.163463 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:33.172031 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff9e71-2a14-42db-942a-23e6dc2109ad None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:33.207208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd91392-e505-4a84-974c-eca7aaf2c34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:33.208261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd91392-e505-4a84-974c-eca7aaf2c34e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:33.208554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd91392-e505-4a84-974c-eca7aaf2c34e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:33.209383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd91392-e505-4a84-974c-eca7aaf2c34e None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:33.210217 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd91392-e505-4a84-974c-eca7aaf2c34e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:33.210530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd91392-e505-4a84-974c-eca7aaf2c34e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:33.211970 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd91392-e505-4a84-974c-eca7aaf2c34e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:33.229995 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 466/920] 10.222.0.81 () {56 vars in 1149 bytes} [Thu Jun 16 18:38:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:33.235467 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b4bf5-19c7-42b0-87c5-e5cb31c8fd46 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:33.339274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61b4bf5-19c7-42b0-87c5-e5cb31c8fd46 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:33.385844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372ed57-c467-4e3b-a23f-dc4d0ab504f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:33.386642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372ed57-c467-4e3b-a23f-dc4d0ab504f3 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:33.386918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372ed57-c467-4e3b-a23f-dc4d0ab504f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:33.387199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372ed57-c467-4e3b-a23f-dc4d0ab504f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:33.397452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372ed57-c467-4e3b-a23f-dc4d0ab504f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:33.398921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372ed57-c467-4e3b-a23f-dc4d0ab504f3 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-9372ed57-c467-4e3b-a23f-dc4d0ab504f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:33.416446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:33.417380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:33.418303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:33.418303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:33.418303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:33.418303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:33.419406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 455/921] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:33.425471 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd91392-e505-4a84-974c-eca7aaf2c34e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:33.528702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd91392-e505-4a84-974c-eca7aaf2c34e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:33.567946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35773a7f-b79d-4ffe-8221-6f4d3ee78358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:33.569035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773a7f-b79d-4ffe-8221-6f4d3ee78358 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:33.569582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773a7f-b79d-4ffe-8221-6f4d3ee78358 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:33.570078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773a7f-b79d-4ffe-8221-6f4d3ee78358 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:33.571024 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35773a7f-b79d-4ffe-8221-6f4d3ee78358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:33.571567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35773a7f-b79d-4ffe-8221-6f4d3ee78358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:33.573196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35773a7f-b79d-4ffe-8221-6f4d3ee78358 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:33.586584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 467/922] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:33.592434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372ed57-c467-4e3b-a23f-dc4d0ab504f3 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:33.649981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:33.704566 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372ed57-c467-4e3b-a23f-dc4d0ab504f3 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:33.742562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6550c4-7fa6-40a3-ae25-446c0470fcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:33.743366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6550c4-7fa6-40a3-ae25-446c0470fcf5 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:33.743652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6550c4-7fa6-40a3-ae25-446c0470fcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:33.744035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6550c4-7fa6-40a3-ae25-446c0470fcf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:33.751294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6550c4-7fa6-40a3-ae25-446c0470fcf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:33.752757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6550c4-7fa6-40a3-ae25-446c0470fcf5 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-fb6550c4-7fa6-40a3-ae25-446c0470fcf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:33.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:33.767490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:33.768368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:33.768368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:33.768368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:33.768368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:33.768368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:33.768368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:33.768368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:33.768368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:33.768368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 456/923] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:33.775176 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35773a7f-b79d-4ffe-8221-6f4d3ee78358 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:33.788777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:33.887335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35773a7f-b79d-4ffe-8221-6f4d3ee78358 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:33.895567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbace3-2b20-4928-939c-8be3f4050287 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:33.895986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbace3-2b20-4928-939c-8be3f4050287 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:33.896223 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbace3-2b20-4928-939c-8be3f4050287 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:33.897725 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 457/924] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:33.902506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812330f-e945-4a56-9add-aa1b85daa5d9 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:33.902795 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812330f-e945-4a56-9add-aa1b85daa5d9 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:33.903072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812330f-e945-4a56-9add-aa1b85daa5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:33.919736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-a812330f-e945-4a56-9add-aa1b85daa5d9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:33.920857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a812330f-e945-4a56-9add-aa1b85daa5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:33.925963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7bb5b-bb95-4577-a083-5c0b16fd6544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:33.926618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7bb5b-bb95-4577-a083-5c0b16fd6544 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:33.926896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7bb5b-bb95-4577-a083-5c0b16fd6544 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:33.927166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7bb5b-bb95-4577-a083-5c0b16fd6544 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:33.927944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7bb5b-bb95-4577-a083-5c0b16fd6544 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:33.928250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7bb5b-bb95-4577-a083-5c0b16fd6544 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:33.929701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7bb5b-bb95-4577-a083-5c0b16fd6544 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:33.944872 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 468/925] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:33.950865 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7bb5b-bb95-4577-a083-5c0b16fd6544 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:34.011453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 458/926] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:34.032033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:34.035519 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812330f-e945-4a56-9add-aa1b85daa5d9 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:34.065815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7bb5b-bb95-4577-a083-5c0b16fd6544 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:34.077121 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:34.107548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27767cc-0eeb-4c44-8c62-fe48a1c6e28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:34.109230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27767cc-0eeb-4c44-8c62-fe48a1c6e28d None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:34.109545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27767cc-0eeb-4c44-8c62-fe48a1c6e28d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:34.109829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27767cc-0eeb-4c44-8c62-fe48a1c6e28d None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/0d78b2dc673341dba7b8cba588471c46/roles/df2b07cfe31149c3990f1f66fdfaed9d` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:34.112980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27767cc-0eeb-4c44-8c62-fe48a1c6e28d None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=0d78b2dc673341dba7b8cba588471c46, role_id=df2b07cfe31149c3990f1f66fdfaed9d)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:34.114748 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27767cc-0eeb-4c44-8c62-fe48a1c6e28d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:34.151046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812330f-e945-4a56-9add-aa1b85daa5d9 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:34.186274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d566d-ff46-4161-a8a4-7d542d126ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:34.187102 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d566d-ff46-4161-a8a4-7d542d126ea3 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:34.187384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d566d-ff46-4161-a8a4-7d542d126ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:34.187655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d566d-ff46-4161-a8a4-7d542d126ea3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:34.188294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d566d-ff46-4161-a8a4-7d542d126ea3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:34.189682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d566d-ff46-4161-a8a4-7d542d126ea3 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-528d566d-ff46-4161-a8a4-7d542d126ea3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:34.201454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:34.203033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 18:38:34.203917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:34.205376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 459/927] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:38:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:34.208841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d566d-ff46-4161-a8a4-7d542d126ea3 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:34.220709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 469/928] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:38:33 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/0d78b2dc673341dba7b8cba588471c46/roles/df2b07cfe31149c3990f1f66fdfaed9d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:38:34.323392 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:34.328411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d566d-ff46-4161-a8a4-7d542d126ea3 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:34.366293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8524f88-59fc-4a5c-b452-2823d9d4eb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:34.367015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8524f88-59fc-4a5c-b452-2823d9d4eb9a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:34.367398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8524f88-59fc-4a5c-b452-2823d9d4eb9a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:34.367675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8524f88-59fc-4a5c-b452-2823d9d4eb9a None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:34.368375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8524f88-59fc-4a5c-b452-2823d9d4eb9a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:34.368681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8524f88-59fc-4a5c-b452-2823d9d4eb9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:34.370091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8524f88-59fc-4a5c-b452-2823d9d4eb9a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:34.377206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 460/929] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:38:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:34.387145 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27767cc-0eeb-4c44-8c62-fe48a1c6e28d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:34.493229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27767cc-0eeb-4c44-8c62-fe48a1c6e28d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:34.523060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:34.529529 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a784246-ef26-4758-9e82-4453fc3e7588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:34.530275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a784246-ef26-4758-9e82-4453fc3e7588 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:34.530709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a784246-ef26-4758-9e82-4453fc3e7588 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:34.531047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a784246-ef26-4758-9e82-4453fc3e7588 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:34.531898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a784246-ef26-4758-9e82-4453fc3e7588 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:34.532275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a784246-ef26-4758-9e82-4453fc3e7588 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:34.533739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a784246-ef26-4758-9e82-4453fc3e7588 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:34.539777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 470/930] 10.222.0.81 () {56 vars in 1114 bytes} [Thu Jun 16 18:38:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:34.545418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8524f88-59fc-4a5c-b452-2823d9d4eb9a None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:34.617396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d3890-bb90-4916-adeb-79cb9daabadd None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:34.617726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d3890-bb90-4916-adeb-79cb9daabadd None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:34.617962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d3890-bb90-4916-adeb-79cb9daabadd None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:34.619702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 471/931] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:34.626177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769ec43-5539-4d25-a652-bfceb0a18f8b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:34.626474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769ec43-5539-4d25-a652-bfceb0a18f8b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:34.626746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9769ec43-5539-4d25-a652-bfceb0a18f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:34.645369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9769ec43-5539-4d25-a652-bfceb0a18f8b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:34.646710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9769ec43-5539-4d25-a652-bfceb0a18f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:34.663076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8524f88-59fc-4a5c-b452-2823d9d4eb9a None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:34.707894 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e497a-7977-4412-a2ab-d710f5b674dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:34.708616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e497a-7977-4412-a2ab-d710f5b674dc None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:34.708908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e497a-7977-4412-a2ab-d710f5b674dc None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:34.709279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236e497a-7977-4412-a2ab-d710f5b674dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:34.710045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e497a-7977-4412-a2ab-d710f5b674dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:34.711420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236e497a-7977-4412-a2ab-d710f5b674dc None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:34.723310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 472/932] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:34.724959 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b35fc3-adf4-48de-b4c1-205fef3a59a0 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:34.725387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b35fc3-adf4-48de-b4c1-205fef3a59a0 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:34.725763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b35fc3-adf4-48de-b4c1-205fef3a59a0 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:34.727247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 473/933] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:34.732276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274db2f-a0dd-4749-b622-9b9c8f959406 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:34.732688 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274db2f-a0dd-4749-b622-9b9c8f959406 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:34.733072 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274db2f-a0dd-4749-b622-9b9c8f959406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:34.750315 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-b274db2f-a0dd-4749-b622-9b9c8f959406 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:34.751448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b274db2f-a0dd-4749-b622-9b9c8f959406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:34.809283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 461/934] 10.222.0.81 () {60 vars in 1144 bytes} [Thu Jun 16 18:38:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:34.809283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e497a-7977-4412-a2ab-d710f5b674dc None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:34.833172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:34.849663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 474/935] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:34.869396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274db2f-a0dd-4749-b622-9b9c8f959406 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:34.924933 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:34.928453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236e497a-7977-4412-a2ab-d710f5b674dc None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:34.966887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6e6b2-0b70-497f-8578-d874311adf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:34.967712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6e6b2-0b70-497f-8578-d874311adf1c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:34.968024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6e6b2-0b70-497f-8578-d874311adf1c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:34.968303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6e6b2-0b70-497f-8578-d874311adf1c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:34.968955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6e6b2-0b70-497f-8578-d874311adf1c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:34.971488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6e6b2-0b70-497f-8578-d874311adf1c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:34.978396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274db2f-a0dd-4749-b622-9b9c8f959406 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-aef6e6b2-0b70-497f-8578-d874311adf1c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:34.991737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:34.992740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:34.993671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 462/936] 10.222.0.81 () {56 vars in 1132 bytes} [Thu Jun 16 18:38:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:35.006509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6e6b2-0b70-497f-8578-d874311adf1c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:35.029339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a14cd9-8be9-4f4a-a24c-5dbe80b1d2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:35.030434 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a14cd9-8be9-4f4a-a24c-5dbe80b1d2cd None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:35.030759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a14cd9-8be9-4f4a-a24c-5dbe80b1d2cd None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:35.031039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a14cd9-8be9-4f4a-a24c-5dbe80b1d2cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:35.031699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a14cd9-8be9-4f4a-a24c-5dbe80b1d2cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:35.033076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a14cd9-8be9-4f4a-a24c-5dbe80b1d2cd None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-e4a14cd9-8be9-4f4a-a24c-5dbe80b1d2cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:35.046886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:35.048156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:35.049198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 475/937] 10.222.0.81 () {56 vars in 1126 bytes} [Thu Jun 16 18:38:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:35.055536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a14cd9-8be9-4f4a-a24c-5dbe80b1d2cd None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:35.100366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:35.118794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6e6b2-0b70-497f-8578-d874311adf1c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:35.154764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabc0a7-8211-48b5-b3ce-ab1a972f47ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:35.156153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabc0a7-8211-48b5-b3ce-ab1a972f47ff None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:35.156489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabc0a7-8211-48b5-b3ce-ab1a972f47ff None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:35.156791 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabc0a7-8211-48b5-b3ce-ab1a972f47ff None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:35.157609 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabc0a7-8211-48b5-b3ce-ab1a972f47ff None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:35.157826 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabc0a7-8211-48b5-b3ce-ab1a972f47ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:35.159232 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabc0a7-8211-48b5-b3ce-ab1a972f47ff None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:35.166780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a14cd9-8be9-4f4a-a24c-5dbe80b1d2cd None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:35.169184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 463/938] 10.222.0.81 () {56 vars in 1126 bytes} [Thu Jun 16 18:38:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:35.170444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabc0a7-8211-48b5-b3ce-ab1a972f47ff None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:35.182419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:35.205003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc5081-2471-4224-8bf5-aaeba30e739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:35.206124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc5081-2471-4224-8bf5-aaeba30e739d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:35.206618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc5081-2471-4224-8bf5-aaeba30e739d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:35.207118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc5081-2471-4224-8bf5-aaeba30e739d None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:35.208073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc5081-2471-4224-8bf5-aaeba30e739d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:35.208650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc5081-2471-4224-8bf5-aaeba30e739d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:35.210345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc5081-2471-4224-8bf5-aaeba30e739d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:35.216860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 476/939] 10.222.0.81 () {56 vars in 1122 bytes} [Thu Jun 16 18:38:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:35.222278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc5081-2471-4224-8bf5-aaeba30e739d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:35.279009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabc0a7-8211-48b5-b3ce-ab1a972f47ff None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:35.315145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816f897-7d01-4a8f-bac5-4239a998d9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:35.315805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816f897-7d01-4a8f-bac5-4239a998d9bb None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:35.316089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816f897-7d01-4a8f-bac5-4239a998d9bb None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:35.316362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816f897-7d01-4a8f-bac5-4239a998d9bb None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:35.317075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816f897-7d01-4a8f-bac5-4239a998d9bb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:35.317412 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816f897-7d01-4a8f-bac5-4239a998d9bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:35.318779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816f897-7d01-4a8f-bac5-4239a998d9bb None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:35.328988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 464/940] 10.222.0.81 () {56 vars in 1126 bytes} [Thu Jun 16 18:38:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:35.329873 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816f897-7d01-4a8f-bac5-4239a998d9bb None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:35.345341 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:35.371691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc5081-2471-4224-8bf5-aaeba30e739d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:35.410846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d50fca-d103-49fb-ab7d-48e8b6667a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:35.411517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50fca-d103-49fb-ab7d-48e8b6667a39 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:35.411800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50fca-d103-49fb-ab7d-48e8b6667a39 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:35.412075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50fca-d103-49fb-ab7d-48e8b6667a39 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:35.412777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d50fca-d103-49fb-ab7d-48e8b6667a39 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:35.413084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d50fca-d103-49fb-ab7d-48e8b6667a39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:35.414789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d50fca-d103-49fb-ab7d-48e8b6667a39 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:35.420325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 477/941] 10.222.0.81 () {56 vars in 1122 bytes} [Thu Jun 16 18:38:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:35.426000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d50fca-d103-49fb-ab7d-48e8b6667a39 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:35.460976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816f897-7d01-4a8f-bac5-4239a998d9bb None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:35.485551 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:35.500064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0737e-86d4-4fdb-9139-b162ca35c69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:35.500757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0737e-86d4-4fdb-9139-b162ca35c69b None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:35.500995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0737e-86d4-4fdb-9139-b162ca35c69b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:35.501286 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0737e-86d4-4fdb-9139-b162ca35c69b None admin] PATH_INFO: `/v3/endpoints` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:35.502324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0737e-86d4-4fdb-9139-b162ca35c69b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:35.503383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0737e-86d4-4fdb-9139-b162ca35c69b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:35.556424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d50fca-d103-49fb-ab7d-48e8b6667a39 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:35.592662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0e769-faeb-4de0-be76-1da592700e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:35.594024 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e769-faeb-4de0-be76-1da592700e14 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:35.594324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e769-faeb-4de0-be76-1da592700e14 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:35.594609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0e769-faeb-4de0-be76-1da592700e14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:35.595397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0e769-faeb-4de0-be76-1da592700e14 None admin] RBAC: Adding query filter params (interface=public, service_id=ac5d822da28f47ea8540e9e4d69acaaf, region_id=RegionOne) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:35.595714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0e769-faeb-4de0-be76-1da592700e14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:35.597116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0e769-faeb-4de0-be76-1da592700e14 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:35.607581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 478/942] 10.222.0.81 () {56 vars in 1251 bytes} [Thu Jun 16 18:38:35 2022] GET /identity/v3/endpoints?service_id=ac5d822da28f47ea8540e9e4d69acaaf&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:35.608567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0e769-faeb-4de0-be76-1da592700e14 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:35.625800 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 465/943] 10.222.0.81 () {60 vars in 1144 bytes} [Thu Jun 16 18:38:35 2022] POST /identity/v3/endpoints => generated 347 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:35.733514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0e769-faeb-4de0-be76-1da592700e14 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:35.741670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:35.791335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60678d5-cc2c-4497-9caa-4877ab08138a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:35.792012 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60678d5-cc2c-4497-9caa-4877ab08138a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:35.792298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60678d5-cc2c-4497-9caa-4877ab08138a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:35.792578 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60678d5-cc2c-4497-9caa-4877ab08138a None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:35.793282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60678d5-cc2c-4497-9caa-4877ab08138a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:35.794720 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60678d5-cc2c-4497-9caa-4877ab08138a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:35.802144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 479/944] 10.222.0.81 () {56 vars in 1087 bytes} [Thu Jun 16 18:38:35 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:38:36.509444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acbff7-ea28-4b7a-8a63-f69d176a672d None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:36.509957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acbff7-ea28-4b7a-8a63-f69d176a672d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:36.510074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5acbff7-ea28-4b7a-8a63-f69d176a672d None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:36.511577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 466/945] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:36.517506 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e612d-4cac-438b-8b86-baaee6cd1578 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:36.517949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e612d-4cac-438b-8b86-baaee6cd1578 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:36.518123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e612d-4cac-438b-8b86-baaee6cd1578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:36.534985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-5b1e612d-4cac-438b-8b86-baaee6cd1578 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:36.536146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5b1e612d-4cac-438b-8b86-baaee6cd1578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:36.609476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 480/946] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:36.632127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acbff7-ea28-4b7a-8a63-f69d176a672d None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:36.699741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:36.736226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5acbff7-ea28-4b7a-8a63-f69d176a672d None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:36.772200 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3de8ac-b64e-4c31-a079-978ec36b1c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:36.773012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3de8ac-b64e-4c31-a079-978ec36b1c05 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:36.773327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3de8ac-b64e-4c31-a079-978ec36b1c05 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:36.773607 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3de8ac-b64e-4c31-a079-978ec36b1c05 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:36.784156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3de8ac-b64e-4c31-a079-978ec36b1c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-9a3de8ac-b64e-4c31-a079-978ec36b1c05 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:36.792423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:36.793383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:36.794270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:36.797503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 467/947] 10.222.0.81 () {56 vars in 1120 bytes} [Thu Jun 16 18:38:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:36.799985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e612d-4cac-438b-8b86-baaee6cd1578 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:36.897347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:36.904103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e612d-4cac-438b-8b86-baaee6cd1578 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:36.938655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9330ed-1569-4255-b5ff-93496f39d700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:36.939707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9330ed-1569-4255-b5ff-93496f39d700 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:36.940207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9330ed-1569-4255-b5ff-93496f39d700 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:36.940673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9330ed-1569-4255-b5ff-93496f39d700 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:36.941648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9330ed-1569-4255-b5ff-93496f39d700 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:36.942155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9330ed-1569-4255-b5ff-93496f39d700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:36.943733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9330ed-1569-4255-b5ff-93496f39d700 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:36.954171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 481/948] 10.222.0.81 () {56 vars in 1115 bytes} [Thu Jun 16 18:38:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:36.964167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3de8ac-b64e-4c31-a079-978ec36b1c05 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:37.052943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d10d1-9c1b-458d-9097-b95308aba943 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.053600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d10d1-9c1b-458d-9097-b95308aba943 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.054083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d10d1-9c1b-458d-9097-b95308aba943 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.056637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 482/949] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:37.062607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc232-cb50-4ce7-8fde-6cf5ba960756 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.062841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc232-cb50-4ce7-8fde-6cf5ba960756 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.063121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc232-cb50-4ce7-8fde-6cf5ba960756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.080044 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-49efc232-cb50-4ce7-8fde-6cf5ba960756 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:37.081253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-49efc232-cb50-4ce7-8fde-6cf5ba960756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:37.081986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3de8ac-b64e-4c31-a079-978ec36b1c05 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:37.110621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:37.120193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd841af-304c-409b-9614-7d9ff57f968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:37.121331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd841af-304c-409b-9614-7d9ff57f968b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.121828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd841af-304c-409b-9614-7d9ff57f968b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.122828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd841af-304c-409b-9614-7d9ff57f968b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.134831 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd841af-304c-409b-9614-7d9ff57f968b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:37.136368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd841af-304c-409b-9614-7d9ff57f968b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:37.161343 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 483/950] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-ebd841af-304c-409b-9614-7d9ff57f968b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:37.161479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:37.162317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:37.163206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:37.163206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:37.163206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:37.163206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:37.163206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 468/951] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:37.168055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efc232-cb50-4ce7-8fde-6cf5ba960756 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:37.184142 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd841af-304c-409b-9614-7d9ff57f968b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:37.263127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:37.275838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efc232-cb50-4ce7-8fde-6cf5ba960756 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:37.292444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd841af-304c-409b-9614-7d9ff57f968b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:37.314129 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3d87a-6652-4f0c-b3c2-d726529470d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:37.315128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3d87a-6652-4f0c-b3c2-d726529470d9 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.315620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3d87a-6652-4f0c-b3c2-d726529470d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.316126 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3d87a-6652-4f0c-b3c2-d726529470d9 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.317120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3d87a-6652-4f0c-b3c2-d726529470d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:37.317651 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3d87a-6652-4f0c-b3c2-d726529470d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:37.319339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3d87a-6652-4f0c-b3c2-d726529470d9 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:37.330869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b692f3-3b3c-4cd5-b4bc-d73d7a6e4383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:37.331539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b692f3-3b3c-4cd5-b4bc-d73d7a6e4383 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.331824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b692f3-3b3c-4cd5-b4bc-d73d7a6e4383 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.332099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b692f3-3b3c-4cd5-b4bc-d73d7a6e4383 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.332788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b692f3-3b3c-4cd5-b4bc-d73d7a6e4383 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:37.333572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 484/952] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:37.338335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b692f3-3b3c-4cd5-b4bc-d73d7a6e4383 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:37.339727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: WARNING keystone.server.flask.common [None req-69b692f3-3b3c-4cd5-b4bc-d73d7a6e4383 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 18:38:37.341481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3d87a-6652-4f0c-b3c2-d726529470d9 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:37.410911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.sql.core [None req-69b692f3-3b3c-4cd5-b4bc-d73d7a6e4383 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 18:38:37.410911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 18:38:37.410911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [parameters: {'id': '8cddb779e52e4f0b91018491773aa753', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 18:38:37.410911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92269) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-69b692f3-3b3c-4cd5-b4bc-d73d7a6e4383 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8cddb779e52e4f0b91018491773aa753.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8cddb779e52e4f0b91018491773aa753. Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:38:37.419175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application result.read() Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 18:38:37.420006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application yield resource Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise self.value Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application session.commit() Jun 16 18:38:37.420973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self.session.flush() Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application compat.raise_( Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 18:38:37.422514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 18:38:37.423423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application result.read() Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:38:37.424336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [parameters: {'id': '8cddb779e52e4f0b91018491773aa753', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:37.425226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:37.426540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:37.428344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:37.428344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 18:38:37.428344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 18:38:37.428344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:37.428344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:37.428344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 18:38:37.428344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 18:38:37.428344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8cddb779e52e4f0b91018491773aa753. Jun 16 18:38:37.428344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:37.428344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 469/953] 10.222.0.81 () {60 vars in 1140 bytes} [Thu Jun 16 18:38:37 2022] POST /identity/v3/projects => generated 262 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:37.430257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b692f3-3b3c-4cd5-b4bc-d73d7a6e4383 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:37.464229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3d87a-6652-4f0c-b3c2-d726529470d9 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:37.466819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:37.504181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ee5be-17e7-4658-9eff-2cc2cb8733ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:37.505367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ee5be-17e7-4658-9eff-2cc2cb8733ec None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.505883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ee5be-17e7-4658-9eff-2cc2cb8733ec None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.506360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ee5be-17e7-4658-9eff-2cc2cb8733ec None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.514806 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ee5be-17e7-4658-9eff-2cc2cb8733ec None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:37.516703 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ee5be-17e7-4658-9eff-2cc2cb8733ec None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-e82ee5be-17e7-4658-9eff-2cc2cb8733ec None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:37.532724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:37.538923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:37.539947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:37.539947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:37.539947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:37.539947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:37.539947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:37.539947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:37.539947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:37.539947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:37.539947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 18:38:37.539947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:37.539947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 485/954] 10.222.0.81 () {56 vars in 1117 bytes} [Thu Jun 16 18:38:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:37.541577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ee5be-17e7-4658-9eff-2cc2cb8733ec None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:37.547566 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b692f3-3b3c-4cd5-b4bc-d73d7a6e4383 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:37.554396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:37.586367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149457ca-ac95-4088-8b48-5e4c48f44ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:37.587611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149457ca-ac95-4088-8b48-5e4c48f44ea7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.588131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149457ca-ac95-4088-8b48-5e4c48f44ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.588628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149457ca-ac95-4088-8b48-5e4c48f44ea7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.596407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149457ca-ac95-4088-8b48-5e4c48f44ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:37.598386 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149457ca-ac95-4088-8b48-5e4c48f44ea7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-149457ca-ac95-4088-8b48-5e4c48f44ea7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:37.615234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:37.616237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:37.617156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:37.617156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:37.617156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:37.617156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:37.617156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:37.617156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:37.617156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:37.617156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:37.625076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 470/955] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:37.626246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149457ca-ac95-4088-8b48-5e4c48f44ea7 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:37.656585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ee5be-17e7-4658-9eff-2cc2cb8733ec None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:37.700719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1c27e-7bc5-4f25-b3b5-0fd249390344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:37.701381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1c27e-7bc5-4f25-b3b5-0fd249390344 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.701676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1c27e-7bc5-4f25-b3b5-0fd249390344 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.701956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1c27e-7bc5-4f25-b3b5-0fd249390344 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.702922 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1c27e-7bc5-4f25-b3b5-0fd249390344 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:37.703183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1c27e-7bc5-4f25-b3b5-0fd249390344 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:37.704620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1c27e-7bc5-4f25-b3b5-0fd249390344 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:37.728928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 486/956] 10.222.0.81 () {56 vars in 1149 bytes} [Thu Jun 16 18:38:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:37.730509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1c27e-7bc5-4f25-b3b5-0fd249390344 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:37.739023 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:37.746489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149457ca-ac95-4088-8b48-5e4c48f44ea7 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:37.791872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc860793-6446-45fa-85ce-3e8dc88363f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:37.792544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc860793-6446-45fa-85ce-3e8dc88363f6 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.792835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc860793-6446-45fa-85ce-3e8dc88363f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.793466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc860793-6446-45fa-85ce-3e8dc88363f6 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.794510 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc860793-6446-45fa-85ce-3e8dc88363f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:37.794827 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc860793-6446-45fa-85ce-3e8dc88363f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:37.796485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc860793-6446-45fa-85ce-3e8dc88363f6 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:37.819083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:37.829277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 471/957] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:37 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:37.865513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1c27e-7bc5-4f25-b3b5-0fd249390344 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:37.907305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876a0d9-be3f-4229-a7cf-5ea8823d0dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:37.907982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876a0d9-be3f-4229-a7cf-5ea8823d0dfd None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.908270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876a0d9-be3f-4229-a7cf-5ea8823d0dfd None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.908550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876a0d9-be3f-4229-a7cf-5ea8823d0dfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.932969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8e95b-8735-4a20-ad69-7180c7ae6c52 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.933329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8e95b-8735-4a20-ad69-7180c7ae6c52 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.933567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8e95b-8735-4a20-ad69-7180c7ae6c52 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.939156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 472/958] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:37.941146 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeb6f5-baca-49ec-9c79-33e721380e42 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:37.941468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeb6f5-baca-49ec-9c79-33e721380e42 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:37.941753 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdeb6f5-baca-49ec-9c79-33e721380e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:37.961054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-abdeb6f5-baca-49ec-9c79-33e721380e42 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:37.962701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-abdeb6f5-baca-49ec-9c79-33e721380e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:37.989032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876a0d9-be3f-4229-a7cf-5ea8823d0dfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:37.990584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876a0d9-be3f-4229-a7cf-5ea8823d0dfd None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:38.056325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 473/959] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:38.057527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50137ad-6b57-4dda-bee0-6f03e07769fc None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:38.057819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50137ad-6b57-4dda-bee0-6f03e07769fc None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:38.058096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50137ad-6b57-4dda-bee0-6f03e07769fc None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:38.060173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 474/960] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:38.065144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:38.067854 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bacd43-b476-4df1-b918-1d8ebbfcd281 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:38.068147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bacd43-b476-4df1-b918-1d8ebbfcd281 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:38.068436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bacd43-b476-4df1-b918-1d8ebbfcd281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:38.086765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-91bacd43-b476-4df1-b918-1d8ebbfcd281 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:38.087971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-91bacd43-b476-4df1-b918-1d8ebbfcd281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:38.109677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 487/961] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:37 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:38:38.110815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876a0d9-be3f-4229-a7cf-5ea8823d0dfd None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:38.167198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 475/962] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:38.174159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bacd43-b476-4df1-b918-1d8ebbfcd281 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:38.192253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:38.215447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876a0d9-be3f-4229-a7cf-5ea8823d0dfd None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:38.248607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b59cf-34d1-4ca2-a148-1a47bbcdcd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:38.250352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b59cf-34d1-4ca2-a148-1a47bbcdcd61 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:38.250725 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b59cf-34d1-4ca2-a148-1a47bbcdcd61 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:38.251095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b59cf-34d1-4ca2-a148-1a47bbcdcd61 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:38.251846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b59cf-34d1-4ca2-a148-1a47bbcdcd61 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:38.253354 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b59cf-34d1-4ca2-a148-1a47bbcdcd61 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-528b59cf-34d1-4ca2-a148-1a47bbcdcd61 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:38.266083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:38.267365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:38:38.268416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:38.270015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 488/963] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:38.270575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b59cf-34d1-4ca2-a148-1a47bbcdcd61 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:38.283309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bacd43-b476-4df1-b918-1d8ebbfcd281 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:38.304022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:38.320465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55828e-4361-4124-90b9-a4832187139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:38.321287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55828e-4361-4124-90b9-a4832187139e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:38.330911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55828e-4361-4124-90b9-a4832187139e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:38.330911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55828e-4361-4124-90b9-a4832187139e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:38.332531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55828e-4361-4124-90b9-a4832187139e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:38.334017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55828e-4361-4124-90b9-a4832187139e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-fd55828e-4361-4124-90b9-a4832187139e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:38.355634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:38.356861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:38.358046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:38.358046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:38.358046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:38.358046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:38.366538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 476/964] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:38.366538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55828e-4361-4124-90b9-a4832187139e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:38.382687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b59cf-34d1-4ca2-a148-1a47bbcdcd61 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:38.424696 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b1aab-9145-44f7-8562-234342ce0e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:38.425871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b1aab-9145-44f7-8562-234342ce0e08 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:38.426305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b1aab-9145-44f7-8562-234342ce0e08 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:38.426587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b1aab-9145-44f7-8562-234342ce0e08 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:38.427236 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b1aab-9145-44f7-8562-234342ce0e08 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:38.428842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b1aab-9145-44f7-8562-234342ce0e08 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:38.434383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-118b1aab-9145-44f7-8562-234342ce0e08 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:38.442569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:38.449909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:38.451049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 489/965] 10.222.0.81 () {56 vars in 1126 bytes} [Thu Jun 16 18:38:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:38.452046 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b1aab-9145-44f7-8562-234342ce0e08 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:38.480739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55828e-4361-4124-90b9-a4832187139e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:38.517772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804af35e-5e9c-4edb-b699-7fb08bc5c04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:38.518442 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804af35e-5e9c-4edb-b699-7fb08bc5c04f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:38.518722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804af35e-5e9c-4edb-b699-7fb08bc5c04f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:38.518996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804af35e-5e9c-4edb-b699-7fb08bc5c04f None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:38.519689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804af35e-5e9c-4edb-b699-7fb08bc5c04f None admin] RBAC: Adding query filter params (name=compute) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:38.519995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804af35e-5e9c-4edb-b699-7fb08bc5c04f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:38.521372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804af35e-5e9c-4edb-b699-7fb08bc5c04f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:38.526669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 477/966] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:38.527433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804af35e-5e9c-4edb-b699-7fb08bc5c04f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:38.551645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:38.558726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b1aab-9145-44f7-8562-234342ce0e08 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:38.595677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444d557-f916-435c-9c05-74c767a62c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:38.596459 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444d557-f916-435c-9c05-74c767a62c58 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:38.596754 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444d557-f916-435c-9c05-74c767a62c58 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:38.597034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444d557-f916-435c-9c05-74c767a62c58 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:38.597784 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444d557-f916-435c-9c05-74c767a62c58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:38.598097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444d557-f916-435c-9c05-74c767a62c58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:38.599474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444d557-f916-435c-9c05-74c767a62c58 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:38.617935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 490/967] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:38.617935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444d557-f916-435c-9c05-74c767a62c58 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:38.634224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804af35e-5e9c-4edb-b699-7fb08bc5c04f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:38.663475 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:38.669747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0d460-aed0-47ce-a31f-e21a747e0c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:38.670404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d460-aed0-47ce-a31f-e21a747e0c05 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:38.670684 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d460-aed0-47ce-a31f-e21a747e0c05 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:38.670947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0d460-aed0-47ce-a31f-e21a747e0c05 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:38.671681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0d460-aed0-47ce-a31f-e21a747e0c05 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:38.671985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0d460-aed0-47ce-a31f-e21a747e0c05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:38.673378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0d460-aed0-47ce-a31f-e21a747e0c05 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:38.682548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 478/968] 10.222.0.81 () {56 vars in 1122 bytes} [Thu Jun 16 18:38:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:38.682548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0d460-aed0-47ce-a31f-e21a747e0c05 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:38.733946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444d557-f916-435c-9c05-74c767a62c58 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:38.772278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bd6f6-b585-49f0-9639-df8104dc275f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:38.772941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bd6f6-b585-49f0-9639-df8104dc275f None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:38.773220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bd6f6-b585-49f0-9639-df8104dc275f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:38.773517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bd6f6-b585-49f0-9639-df8104dc275f None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:38.774216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bd6f6-b585-49f0-9639-df8104dc275f None admin] RBAC: Adding query filter params (type=compute) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:38.774521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bd6f6-b585-49f0-9639-df8104dc275f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:38.775873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bd6f6-b585-49f0-9639-df8104dc275f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:38.785276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 491/969] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:38 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:38.785276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bd6f6-b585-49f0-9639-df8104dc275f None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:38.793961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0d460-aed0-47ce-a31f-e21a747e0c05 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:38.811193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:38.846131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a47457-cfa8-435f-be79-7e9010e9fc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:38.847277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a47457-cfa8-435f-be79-7e9010e9fc81 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:38.847579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a47457-cfa8-435f-be79-7e9010e9fc81 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:38.847857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a47457-cfa8-435f-be79-7e9010e9fc81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:38.858516 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a47457-cfa8-435f-be79-7e9010e9fc81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:38.860866 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a47457-cfa8-435f-be79-7e9010e9fc81 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-d3a47457-cfa8-435f-be79-7e9010e9fc81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:38.881339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:38.882498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:38.883424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:38.883424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:38.883424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:38.883424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:38.883424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:38.883424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:38.883424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:38.883424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:38.883424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 479/970] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:38.888395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a47457-cfa8-435f-be79-7e9010e9fc81 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:38.903770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bd6f6-b585-49f0-9639-df8104dc275f None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:38.941828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542b691-86f3-4ee3-b40f-1caaf0ed5103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:38.942714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b691-86f3-4ee3-b40f-1caaf0ed5103 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:38.943093 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b691-86f3-4ee3-b40f-1caaf0ed5103 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:38.943460 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b691-86f3-4ee3-b40f-1caaf0ed5103 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:38.944319 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542b691-86f3-4ee3-b40f-1caaf0ed5103 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:38.944721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542b691-86f3-4ee3-b40f-1caaf0ed5103 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:38.946245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542b691-86f3-4ee3-b40f-1caaf0ed5103 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:38.957117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 492/971] 10.222.0.81 () {56 vars in 1122 bytes} [Thu Jun 16 18:38:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:38.958928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542b691-86f3-4ee3-b40f-1caaf0ed5103 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:38.960659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:39.020339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a47457-cfa8-435f-be79-7e9010e9fc81 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:39.060802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e2ab5-3186-48fe-a173-6a6535d17573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:39.061647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e2ab5-3186-48fe-a173-6a6535d17573 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:39.062037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e2ab5-3186-48fe-a173-6a6535d17573 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:39.062940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e2ab5-3186-48fe-a173-6a6535d17573 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:39.063925 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e2ab5-3186-48fe-a173-6a6535d17573 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:39.064331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e2ab5-3186-48fe-a173-6a6535d17573 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:39.065882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e2ab5-3186-48fe-a173-6a6535d17573 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:39.067835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:39.079796 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 480/972] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:39.086042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e2ab5-3186-48fe-a173-6a6535d17573 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:39.091063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542b691-86f3-4ee3-b40f-1caaf0ed5103 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:39.138410 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371b6fc-58aa-4678-86c4-c34921e2568e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:39.139614 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371b6fc-58aa-4678-86c4-c34921e2568e None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:39.141127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371b6fc-58aa-4678-86c4-c34921e2568e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:39.141600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371b6fc-58aa-4678-86c4-c34921e2568e None admin] PATH_INFO: `/v3/endpoints` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:39.142756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371b6fc-58aa-4678-86c4-c34921e2568e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:39.144964 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371b6fc-58aa-4678-86c4-c34921e2568e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:39.195633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e2ab5-3186-48fe-a173-6a6535d17573 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:39.210646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:39.230232 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27970af-0786-42c8-bdf6-456737e53558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:39.231054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27970af-0786-42c8-bdf6-456737e53558 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:39.231394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27970af-0786-42c8-bdf6-456737e53558 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:39.231724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27970af-0786-42c8-bdf6-456737e53558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:39.237471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27970af-0786-42c8-bdf6-456737e53558 None admin] RBAC: Adding query filter params (role.id=df2b07cfe31149c3990f1f66fdfaed9d, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=0d78b2dc673341dba7b8cba588471c46) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:39.237471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27970af-0786-42c8-bdf6-456737e53558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:39.237471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27970af-0786-42c8-bdf6-456737e53558 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:39.237934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 493/973] 10.222.0.81 () {60 vars in 1144 bytes} [Thu Jun 16 18:38:38 2022] POST /identity/v3/endpoints => generated 349 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:39.243244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 481/974] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:39 2022] GET /identity/v3/role_assignments?user.id=0d78b2dc673341dba7b8cba588471c46&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=df2b07cfe31149c3990f1f66fdfaed9d => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:40.101901 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b1d4c-7d37-4ea5-87c9-71c5e50e93db None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:40.102420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b1d4c-7d37-4ea5-87c9-71c5e50e93db None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:40.102499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b1d4c-7d37-4ea5-87c9-71c5e50e93db None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:40.104071 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 494/975] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:40.109562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71ad77-4087-40cc-98a1-13e9b8fe9ecf None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:40.109958 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71ad77-4087-40cc-98a1-13e9b8fe9ecf None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:40.110135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71ad77-4087-40cc-98a1-13e9b8fe9ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:40.127888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ed71ad77-4087-40cc-98a1-13e9b8fe9ecf None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:40.129074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ed71ad77-4087-40cc-98a1-13e9b8fe9ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:40.209373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 482/976] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:40.228906 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b1d4c-7d37-4ea5-87c9-71c5e50e93db None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:40.333083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b1d4c-7d37-4ea5-87c9-71c5e50e93db None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:40.370056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e035831-8db8-483f-97c4-0e26e5d138a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:40.370759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e035831-8db8-483f-97c4-0e26e5d138a7 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:40.371054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e035831-8db8-483f-97c4-0e26e5d138a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:40.371327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e035831-8db8-483f-97c4-0e26e5d138a7 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:40.372120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e035831-8db8-483f-97c4-0e26e5d138a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:40.373802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e035831-8db8-483f-97c4-0e26e5d138a7 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:40.375441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: WARNING keystone.server.flask.common [None req-0e035831-8db8-483f-97c4-0e26e5d138a7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 18:38:40.389780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:40.424090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.sql.core [None req-0e035831-8db8-483f-97c4-0e26e5d138a7 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 18:38:40.424090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 18:38:40.424090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [parameters: {'user_id': '7b1f4b181fcc4287b2d6d1bf7f1fcd96', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 18:38:40.424090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92270) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-0e035831-8db8-483f-97c4-0e26e5d138a7 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application result.read() Jun 16 18:38:40.442797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:38:40.443911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application yield resource Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application next(self.gen) Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise self.value Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application session.commit() Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 18:38:40.445121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self.session.flush() Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application compat.raise_( Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 18:38:40.446544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 18:38:40.447650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application conn.query(q) Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application result.read() Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 18:38:40.448532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [parameters: {'user_id': '7b1f4b181fcc4287b2d6d1bf7f1fcd96', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:40.449476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 18:38:40.450542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 18:38:40.452075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 18:38:40.452075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 18:38:40.452075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 18:38:40.452075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 18:38:40.452075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 18:38:40.452075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:40.458574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 495/977] 10.222.0.81 () {60 vars in 1131 bytes} [Thu Jun 16 18:38:40 2022] POST /identity/v3/users => generated 160 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:40.461701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71ad77-4087-40cc-98a1-13e9b8fe9ecf None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:40.573579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71ad77-4087-40cc-98a1-13e9b8fe9ecf None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:40.575896 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:40.607569 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8f18f-0268-4861-add2-7468f35487a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:40.608373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8f18f-0268-4861-add2-7468f35487a1 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:40.608658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8f18f-0268-4861-add2-7468f35487a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:40.608934 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8f18f-0268-4861-add2-7468f35487a1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:40.616752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8f18f-0268-4861-add2-7468f35487a1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:40.618197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8f18f-0268-4861-add2-7468f35487a1 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-d5a8f18f-0268-4861-add2-7468f35487a1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:40.633389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:40.634575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:40.635830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:40.635830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:40.635830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:40.635830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:40.635830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:40.635830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:40.635830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:40.635830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:40.635830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 18:38:40.635830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:40.635830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 483/978] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:38:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:40.643192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e035831-8db8-483f-97c4-0e26e5d138a7 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:40.750783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e035831-8db8-483f-97c4-0e26e5d138a7 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:40.786115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba3b8c-228b-49a1-b0f2-718d1720aaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:40.787180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba3b8c-228b-49a1-b0f2-718d1720aaba None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:40.787679 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba3b8c-228b-49a1-b0f2-718d1720aaba None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:40.788159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba3b8c-228b-49a1-b0f2-718d1720aaba None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:40.789161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba3b8c-228b-49a1-b0f2-718d1720aaba None admin] RBAC: Adding query filter params (name=cinder) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:40.789701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba3b8c-228b-49a1-b0f2-718d1720aaba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:40.791545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba3b8c-228b-49a1-b0f2-718d1720aaba None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:40.836522 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 496/979] 10.222.0.81 () {56 vars in 1101 bytes} [Thu Jun 16 18:38:40 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:41.067615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf15b2-eeea-4e89-8d09-b3d5ffe9c338 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:41.067958 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf15b2-eeea-4e89-8d09-b3d5ffe9c338 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:41.068244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf15b2-eeea-4e89-8d09-b3d5ffe9c338 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:41.069924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 484/980] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:41.075355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781346f-cd81-4e9a-a9ea-9503b8bf3971 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:41.075652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781346f-cd81-4e9a-a9ea-9503b8bf3971 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:41.075936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781346f-cd81-4e9a-a9ea-9503b8bf3971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:41.094121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-8781346f-cd81-4e9a-a9ea-9503b8bf3971 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:41.095628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8781346f-cd81-4e9a-a9ea-9503b8bf3971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:41.150656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:41.165326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 497/981] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:41.188539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf15b2-eeea-4e89-8d09-b3d5ffe9c338 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:41.288929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf15b2-eeea-4e89-8d09-b3d5ffe9c338 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:41.322732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a1db6-695b-4cda-846f-a0a895b2bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:41.323318 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a1db6-695b-4cda-846f-a0a895b2bd0a None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:41.323602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a1db6-695b-4cda-846f-a0a895b2bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:41.323878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a1db6-695b-4cda-846f-a0a895b2bd0a None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:41.324593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a1db6-695b-4cda-846f-a0a895b2bd0a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:41.326002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a1db6-695b-4cda-846f-a0a895b2bd0a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:41.401333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 485/982] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:38:41 2022] POST /identity/v3/services => generated 242 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:41.412620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceeddc-f334-48a2-9ad5-b2a94213c5b3 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:41.413016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceeddc-f334-48a2-9ad5-b2a94213c5b3 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:41.413185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ceeddc-f334-48a2-9ad5-b2a94213c5b3 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:41.414998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 498/983] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:41.421029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321359b-8ecf-45fb-8a57-30a13d12d509 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:41.421408 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321359b-8ecf-45fb-8a57-30a13d12d509 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:41.421649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321359b-8ecf-45fb-8a57-30a13d12d509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:41.441141 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-8321359b-8ecf-45fb-8a57-30a13d12d509 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:41.442476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8321359b-8ecf-45fb-8a57-30a13d12d509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:41.463634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:41.488061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d6974-9d48-4df0-81c7-340fad638969 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:41.488421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d6974-9d48-4df0-81c7-340fad638969 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:41.488633 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d6974-9d48-4df0-81c7-340fad638969 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:41.490109 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 499/984] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:41.499661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a1bee-d066-49af-9e82-c8bcdffc8cf4 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:41.500082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a1bee-d066-49af-9e82-c8bcdffc8cf4 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:41.500374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a1bee-d066-49af-9e82-c8bcdffc8cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:41.519467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-609a1bee-d066-49af-9e82-c8bcdffc8cf4 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:41.520680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-609a1bee-d066-49af-9e82-c8bcdffc8cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:41.541482 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 486/985] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:41.560067 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321359b-8ecf-45fb-8a57-30a13d12d509 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:41.600328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 500/986] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:41.623885 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a1bee-d066-49af-9e82-c8bcdffc8cf4 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:41.678564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321359b-8ecf-45fb-8a57-30a13d12d509 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:41.715459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c35361-7d1d-4036-86a9-4adb191bb516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:41.716327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c35361-7d1d-4036-86a9-4adb191bb516 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:41.716641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c35361-7d1d-4036-86a9-4adb191bb516 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:41.716919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c35361-7d1d-4036-86a9-4adb191bb516 None admin] PATH_INFO: `/v3/services/image` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:41.717603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c35361-7d1d-4036-86a9-4adb191bb516 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:41.719422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c35361-7d1d-4036-86a9-4adb191bb516 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:41.728057 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a1bee-d066-49af-9e82-c8bcdffc8cf4 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-87c35361-7d1d-4036-86a9-4adb191bb516 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:41.731541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:41.733062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:38:41.734537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:41.738303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 487/987] 10.222.0.81 () {56 vars in 1105 bytes} [Thu Jun 16 18:38:41 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:41.738984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c35361-7d1d-4036-86a9-4adb191bb516 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:41.746518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:41.757349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:41.766097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e1ec5-91fb-49c8-8b0d-0eabbde7c1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:41.766925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e1ec5-91fb-49c8-8b0d-0eabbde7c1d6 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:41.767183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e1ec5-91fb-49c8-8b0d-0eabbde7c1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:41.767455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e1ec5-91fb-49c8-8b0d-0eabbde7c1d6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:41.768107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e1ec5-91fb-49c8-8b0d-0eabbde7c1d6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:41.769785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e1ec5-91fb-49c8-8b0d-0eabbde7c1d6 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-113e1ec5-91fb-49c8-8b0d-0eabbde7c1d6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:41.784642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:41.789034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:41.790196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 501/988] 10.222.0.81 () {56 vars in 1126 bytes} [Thu Jun 16 18:38:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:41.791720 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e1ec5-91fb-49c8-8b0d-0eabbde7c1d6 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:41.847096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c35361-7d1d-4036-86a9-4adb191bb516 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:41.887241 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870650f1-299b-4574-b745-eec1bacc0d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:41.887831 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870650f1-299b-4574-b745-eec1bacc0d6d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:41.888172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870650f1-299b-4574-b745-eec1bacc0d6d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:41.888435 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870650f1-299b-4574-b745-eec1bacc0d6d None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:41.889087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870650f1-299b-4574-b745-eec1bacc0d6d None admin] RBAC: Adding query filter params (name=image) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:41.889433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870650f1-299b-4574-b745-eec1bacc0d6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:41.890799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870650f1-299b-4574-b745-eec1bacc0d6d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:41.896017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 488/989] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:41.897777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e1ec5-91fb-49c8-8b0d-0eabbde7c1d6 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:41.901381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870650f1-299b-4574-b745-eec1bacc0d6d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:41.936596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6527d-1657-4da2-b568-588562a4cba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:41.937324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6527d-1657-4da2-b568-588562a4cba6 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:41.937657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6527d-1657-4da2-b568-588562a4cba6 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:41.937937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6527d-1657-4da2-b568-588562a4cba6 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:41.938640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6527d-1657-4da2-b568-588562a4cba6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:41.938951 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6527d-1657-4da2-b568-588562a4cba6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:41.940330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6527d-1657-4da2-b568-588562a4cba6 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:41.949805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 502/990] 10.222.0.81 () {56 vars in 1122 bytes} [Thu Jun 16 18:38:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:41.951027 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6527d-1657-4da2-b568-588562a4cba6 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:41.981229 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:42.001746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:42.007280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870650f1-299b-4574-b745-eec1bacc0d6d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:42.042821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc47a3-a99b-47de-8507-2f169c29eb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:42.043484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc47a3-a99b-47de-8507-2f169c29eb99 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:42.044323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc47a3-a99b-47de-8507-2f169c29eb99 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:42.044600 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc47a3-a99b-47de-8507-2f169c29eb99 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:42.045319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc47a3-a99b-47de-8507-2f169c29eb99 None admin] RBAC: Adding query filter params (type=image) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:42.045627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc47a3-a99b-47de-8507-2f169c29eb99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:42.046980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc47a3-a99b-47de-8507-2f169c29eb99 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:42.051555 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 489/991] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:42.060381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6527d-1657-4da2-b568-588562a4cba6 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:42.098399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957cef6-feed-4b27-859d-a48b16d28fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:42.099337 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957cef6-feed-4b27-859d-a48b16d28fe4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:42.099793 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957cef6-feed-4b27-859d-a48b16d28fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:42.100219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957cef6-feed-4b27-859d-a48b16d28fe4 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:42.101124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957cef6-feed-4b27-859d-a48b16d28fe4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:42.101633 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957cef6-feed-4b27-859d-a48b16d28fe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:42.103207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957cef6-feed-4b27-859d-a48b16d28fe4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:42.109896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 503/992] 10.222.0.81 () {56 vars in 1122 bytes} [Thu Jun 16 18:38:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:42.115867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc47a3-a99b-47de-8507-2f169c29eb99 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:42.225760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc47a3-a99b-47de-8507-2f169c29eb99 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:42.260025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e1038-36ad-490d-bb15-dff152da2d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:42.260685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e1038-36ad-490d-bb15-dff152da2d6c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:42.260964 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e1038-36ad-490d-bb15-dff152da2d6c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:42.261262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e1038-36ad-490d-bb15-dff152da2d6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:42.262040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e1038-36ad-490d-bb15-dff152da2d6c None admin] RBAC: Adding query filter params (interface=admin, service_id=ac5d822da28f47ea8540e9e4d69acaaf, region_id=RegionOne) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:42.262354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e1038-36ad-490d-bb15-dff152da2d6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:42.263704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e1038-36ad-490d-bb15-dff152da2d6c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:42.268854 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 490/993] 10.222.0.81 () {56 vars in 1249 bytes} [Thu Jun 16 18:38:42 2022] GET /identity/v3/endpoints?service_id=ac5d822da28f47ea8540e9e4d69acaaf&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:42.274955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957cef6-feed-4b27-859d-a48b16d28fe4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:42.381650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957cef6-feed-4b27-859d-a48b16d28fe4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:42.392846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:42.415910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b1a0b-d977-495e-99bb-c18f25d899ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:42.416569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1a0b-d977-495e-99bb-c18f25d899ee None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:42.416857 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1a0b-d977-495e-99bb-c18f25d899ee None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:42.417132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b1a0b-d977-495e-99bb-c18f25d899ee None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:42.417867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b1a0b-d977-495e-99bb-c18f25d899ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:42.419296 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b1a0b-d977-495e-99bb-c18f25d899ee None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:42.430273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 504/994] 10.222.0.81 () {56 vars in 1087 bytes} [Thu Jun 16 18:38:42 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:38:43.575724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ac559-4634-48d3-8de0-7faebaf44fb1 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:43.576259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ac559-4634-48d3-8de0-7faebaf44fb1 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:43.576338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ac559-4634-48d3-8de0-7faebaf44fb1 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:43.577728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 491/995] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:43.582795 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a07ade-0b70-4fe5-b6c1-0ee47a8040ab None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:43.583216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a07ade-0b70-4fe5-b6c1-0ee47a8040ab None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:43.583404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a07ade-0b70-4fe5-b6c1-0ee47a8040ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:43.599853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-87a07ade-0b70-4fe5-b6c1-0ee47a8040ab None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:43.601039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-87a07ade-0b70-4fe5-b6c1-0ee47a8040ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:43.670116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 505/996] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:43.693364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ac559-4634-48d3-8de0-7faebaf44fb1 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:43.702956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:43.795281 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ac559-4634-48d3-8de0-7faebaf44fb1 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:43.830271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13872b42-e3e3-419a-8f87-2717f64ac3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:43.831066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13872b42-e3e3-419a-8f87-2717f64ac3b3 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:43.831352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13872b42-e3e3-419a-8f87-2717f64ac3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:43.831625 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13872b42-e3e3-419a-8f87-2717f64ac3b3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:43.832262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13872b42-e3e3-419a-8f87-2717f64ac3b3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:43.833678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13872b42-e3e3-419a-8f87-2717f64ac3b3 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-13872b42-e3e3-419a-8f87-2717f64ac3b3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:43.845372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:43.846359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:38:43.847191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:43.849395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 492/997] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:43.852848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a07ade-0b70-4fe5-b6c1-0ee47a8040ab None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:43.954271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a07ade-0b70-4fe5-b6c1-0ee47a8040ab None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:43.956479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:43.988397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbd25b-b55f-41c9-bd62-3ebe7b61072c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:43.989054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbd25b-b55f-41c9-bd62-3ebe7b61072c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:43.989371 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbd25b-b55f-41c9-bd62-3ebe7b61072c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:43.989646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbd25b-b55f-41c9-bd62-3ebe7b61072c None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:43.990334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbd25b-b55f-41c9-bd62-3ebe7b61072c None admin] RBAC: Adding query filter params (name=compute) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:43.990641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbd25b-b55f-41c9-bd62-3ebe7b61072c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:43.992811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbd25b-b55f-41c9-bd62-3ebe7b61072c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:44.002375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 506/998] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:44.003504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13872b42-e3e3-419a-8f87-2717f64ac3b3 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:44.072124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:44.106584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13872b42-e3e3-419a-8f87-2717f64ac3b3 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:44.142919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642aaad4-b5c2-47d6-8b2a-cf91b0a79531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:44.143605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aaad4-b5c2-47d6-8b2a-cf91b0a79531 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:44.143891 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aaad4-b5c2-47d6-8b2a-cf91b0a79531 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:44.144169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aaad4-b5c2-47d6-8b2a-cf91b0a79531 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:44.144883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642aaad4-b5c2-47d6-8b2a-cf91b0a79531 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:44.145195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642aaad4-b5c2-47d6-8b2a-cf91b0a79531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:44.146780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642aaad4-b5c2-47d6-8b2a-cf91b0a79531 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:44.157716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 493/999] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:44.159015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbd25b-b55f-41c9-bd62-3ebe7b61072c None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:44.192143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27438ce-2174-4aa1-8f0c-853854f362c0 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:44.192429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27438ce-2174-4aa1-8f0c-853854f362c0 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:44.192708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27438ce-2174-4aa1-8f0c-853854f362c0 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:44.195816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 494/1000] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:44.199205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e950f-2389-4d42-84a9-9698d18a91cb None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:44.199496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e950f-2389-4d42-84a9-9698d18a91cb None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:44.199778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e950f-2389-4d42-84a9-9698d18a91cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:44.217084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-216e950f-2389-4d42-84a9-9698d18a91cb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:44.218231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-216e950f-2389-4d42-84a9-9698d18a91cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:44.262650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbd25b-b55f-41c9-bd62-3ebe7b61072c None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:44.288741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 495/1001] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:44.300495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b370f-9f0f-41cc-88e1-4c4bfc460341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:44.301354 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b370f-9f0f-41cc-88e1-4c4bfc460341 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:44.301808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b370f-9f0f-41cc-88e1-4c4bfc460341 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:44.302234 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b370f-9f0f-41cc-88e1-4c4bfc460341 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:44.303204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b370f-9f0f-41cc-88e1-4c4bfc460341 None admin] RBAC: Adding query filter params (interface=public, service_id=29c4bf6558334f539c240a70a38e9848, region_id=RegionOne) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:44.303664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b370f-9f0f-41cc-88e1-4c4bfc460341 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:44.305175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b370f-9f0f-41cc-88e1-4c4bfc460341 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:44.310452 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 507/1002] 10.222.0.81 () {56 vars in 1251 bytes} [Thu Jun 16 18:38:44 2022] GET /identity/v3/endpoints?service_id=29c4bf6558334f539c240a70a38e9848&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:44.311246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e950f-2389-4d42-84a9-9698d18a91cb None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:44.316991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b370f-9f0f-41cc-88e1-4c4bfc460341 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:44.354575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:44.384735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:44.424015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e950f-2389-4d42-84a9-9698d18a91cb None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:44.430719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b370f-9f0f-41cc-88e1-4c4bfc460341 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:44.458780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9ae03-533b-4c81-af8d-806ef21c21f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:44.459973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9ae03-533b-4c81-af8d-806ef21c21f5 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:44.460257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9ae03-533b-4c81-af8d-806ef21c21f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:44.460533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9ae03-533b-4c81-af8d-806ef21c21f5 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:44.461272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9ae03-533b-4c81-af8d-806ef21c21f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:44.462631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9ae03-533b-4c81-af8d-806ef21c21f5 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:44.467782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a39f5f-68eb-4d91-bdd7-5f614d4af885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:44.468477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a39f5f-68eb-4d91-bdd7-5f614d4af885 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:44.468764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a39f5f-68eb-4d91-bdd7-5f614d4af885 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:44.469051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a39f5f-68eb-4d91-bdd7-5f614d4af885 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:44.469764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a39f5f-68eb-4d91-bdd7-5f614d4af885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:44.471177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a39f5f-68eb-4d91-bdd7-5f614d4af885 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:44.478358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 508/1003] 10.222.0.81 () {56 vars in 1087 bytes} [Thu Jun 16 18:38:44 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:38:44.542282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 496/1004] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:38:44 2022] POST /identity/v3/services => generated 242 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:38:44.550363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683daf70-cd65-416f-ae7f-45707ca72851 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:44.550724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683daf70-cd65-416f-ae7f-45707ca72851 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:44.550926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683daf70-cd65-416f-ae7f-45707ca72851 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:44.553180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 509/1005] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:44.559446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d39c9-e53c-4534-8341-ad7233ecc626 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:44.559814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d39c9-e53c-4534-8341-ad7233ecc626 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:44.560028 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d39c9-e53c-4534-8341-ad7233ecc626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:44.579364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b70d39c9-e53c-4534-8341-ad7233ecc626 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:44.580632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b70d39c9-e53c-4534-8341-ad7233ecc626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:44.667612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 497/1006] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:44 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:44.691684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683daf70-cd65-416f-ae7f-45707ca72851 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:44.791972 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:44.798313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683daf70-cd65-416f-ae7f-45707ca72851 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:44.832232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe2dc9-4d9b-4870-8dc4-1fe8d9c694cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:44.833066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe2dc9-4d9b-4870-8dc4-1fe8d9c694cb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:44.833375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe2dc9-4d9b-4870-8dc4-1fe8d9c694cb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:44.833651 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe2dc9-4d9b-4870-8dc4-1fe8d9c694cb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:44.834300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe2dc9-4d9b-4870-8dc4-1fe8d9c694cb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:44.835659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe2dc9-4d9b-4870-8dc4-1fe8d9c694cb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-9bfe2dc9-4d9b-4870-8dc4-1fe8d9c694cb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:44.847649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:44.849530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 18:38:44.850418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:44.851616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 510/1007] 10.222.0.81 () {56 vars in 1126 bytes} [Thu Jun 16 18:38:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:44.855639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d39c9-e53c-4534-8341-ad7233ecc626 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:44.906484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:44.955817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d39c9-e53c-4534-8341-ad7233ecc626 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:44.993547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5723b-46e2-4314-8f13-fd67e3e98b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:44.994217 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5723b-46e2-4314-8f13-fd67e3e98b42 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:44.994502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5723b-46e2-4314-8f13-fd67e3e98b42 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:44.994895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5723b-46e2-4314-8f13-fd67e3e98b42 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:44.995597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5723b-46e2-4314-8f13-fd67e3e98b42 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:44.995904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5723b-46e2-4314-8f13-fd67e3e98b42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:44.997317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5723b-46e2-4314-8f13-fd67e3e98b42 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:45.002305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 498/1008] 10.222.0.81 () {56 vars in 1122 bytes} [Thu Jun 16 18:38:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:45.008140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe2dc9-4d9b-4870-8dc4-1fe8d9c694cb None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:45.108165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe2dc9-4d9b-4870-8dc4-1fe8d9c694cb None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:45.140527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e2c3f-ed7d-49b9-bd73-9fd581a87e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:45.141302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e2c3f-ed7d-49b9-bd73-9fd581a87e58 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:45.141662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e2c3f-ed7d-49b9-bd73-9fd581a87e58 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:45.142008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e2c3f-ed7d-49b9-bd73-9fd581a87e58 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:45.142776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e2c3f-ed7d-49b9-bd73-9fd581a87e58 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:45.143158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e2c3f-ed7d-49b9-bd73-9fd581a87e58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:45.145118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e2c3f-ed7d-49b9-bd73-9fd581a87e58 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:45.150282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 511/1009] 10.222.0.81 () {56 vars in 1122 bytes} [Thu Jun 16 18:38:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:45.155851 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5723b-46e2-4314-8f13-fd67e3e98b42 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:45.256581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5723b-46e2-4314-8f13-fd67e3e98b42 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:45.271364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:45.291599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad745e77-b49e-4925-87a6-90b6ecec4999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:45.292345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad745e77-b49e-4925-87a6-90b6ecec4999 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:45.292695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad745e77-b49e-4925-87a6-90b6ecec4999 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:45.293035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad745e77-b49e-4925-87a6-90b6ecec4999 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:45.293869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad745e77-b49e-4925-87a6-90b6ecec4999 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:45.295358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad745e77-b49e-4925-87a6-90b6ecec4999 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:45.381287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 499/1010] 10.222.0.81 () {60 vars in 1144 bytes} [Thu Jun 16 18:38:45 2022] POST /identity/v3/endpoints => generated 325 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:38:46.662769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e35479-53f6-475a-bd94-ad2984685fba None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:46.662769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e35479-53f6-475a-bd94-ad2984685fba None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:46.662769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e35479-53f6-475a-bd94-ad2984685fba None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:46.662769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 512/1011] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:46.667083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2441c70-f0db-4b5b-8a2a-da3b0abbb167 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:46.667456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2441c70-f0db-4b5b-8a2a-da3b0abbb167 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:46.667642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2441c70-f0db-4b5b-8a2a-da3b0abbb167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:46.684617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d2441c70-f0db-4b5b-8a2a-da3b0abbb167 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:46.685843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d2441c70-f0db-4b5b-8a2a-da3b0abbb167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:46.692193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0eb74-37a3-4a80-921b-ba24326c8880 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:46.692484 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0eb74-37a3-4a80-921b-ba24326c8880 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:46.692849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0eb74-37a3-4a80-921b-ba24326c8880 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:46.694162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 513/1012] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:46.698880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be9910-7364-4173-87e4-aee043d9a5bf None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:46.699169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be9910-7364-4173-87e4-aee043d9a5bf None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:46.699456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be9910-7364-4173-87e4-aee043d9a5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:46.716584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-c9be9910-7364-4173-87e4-aee043d9a5bf None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:46.717838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c9be9910-7364-4173-87e4-aee043d9a5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:46.736579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:46.763585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 500/1013] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:46 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:46.785414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 514/1014] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:46 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:46.786499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2441c70-f0db-4b5b-8a2a-da3b0abbb167 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:46.807789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be9910-7364-4173-87e4-aee043d9a5bf None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:46.888365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:46.893286 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2441c70-f0db-4b5b-8a2a-da3b0abbb167 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:46.910285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be9910-7364-4173-87e4-aee043d9a5bf None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:46.928277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e142555-898b-4f0b-8ff2-36ee4bba5639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:46.929382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142555-898b-4f0b-8ff2-36ee4bba5639 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:46.929846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142555-898b-4f0b-8ff2-36ee4bba5639 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:46.930421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e142555-898b-4f0b-8ff2-36ee4bba5639 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:46.931071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e142555-898b-4f0b-8ff2-36ee4bba5639 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:46.932445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e142555-898b-4f0b-8ff2-36ee4bba5639 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:46.944739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d73ab-7bb3-4f6a-9657-240dec9907ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-7e142555-898b-4f0b-8ff2-36ee4bba5639 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:46.944911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:46.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 18:38:46.946788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:46.947516 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d73ab-7bb3-4f6a-9657-240dec9907ee None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:46.947516 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d73ab-7bb3-4f6a-9657-240dec9907ee None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:46.947516 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d73ab-7bb3-4f6a-9657-240dec9907ee None admin] PATH_INFO: `/v3/services/image` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:46.947516 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d73ab-7bb3-4f6a-9657-240dec9907ee None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:46.949045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d73ab-7bb3-4f6a-9657-240dec9907ee None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:46.950966 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 501/1015] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:46.951936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e142555-898b-4f0b-8ff2-36ee4bba5639 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-487d73ab-7bb3-4f6a-9657-240dec9907ee None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:46.961779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:46.962878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:46.963710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 515/1016] 10.222.0.81 () {56 vars in 1105 bytes} [Thu Jun 16 18:38:46 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:46.968763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d73ab-7bb3-4f6a-9657-240dec9907ee None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:46.989282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:47.061902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e142555-898b-4f0b-8ff2-36ee4bba5639 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:47.073174 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d73ab-7bb3-4f6a-9657-240dec9907ee None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:47.096204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e4c85-1d84-499c-860a-a60235d45b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:47.097137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e4c85-1d84-499c-860a-a60235d45b7d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:47.097629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e4c85-1d84-499c-860a-a60235d45b7d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:47.098021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e4c85-1d84-499c-860a-a60235d45b7d None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:47.098611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e4c85-1d84-499c-860a-a60235d45b7d None admin] RBAC: Adding query filter params (name=compute) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:47.098922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e4c85-1d84-499c-860a-a60235d45b7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:47.100293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e4c85-1d84-499c-860a-a60235d45b7d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:47.105223 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 502/1017] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:47.108384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604f2f4-59b9-41d1-8930-5be565751ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:47.109224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604f2f4-59b9-41d1-8930-5be565751ca6 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:47.109544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604f2f4-59b9-41d1-8930-5be565751ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:47.109821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604f2f4-59b9-41d1-8930-5be565751ca6 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:47.110579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604f2f4-59b9-41d1-8930-5be565751ca6 None admin] RBAC: Adding query filter params (name=image) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:47.110892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604f2f4-59b9-41d1-8930-5be565751ca6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:47.111376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e4c85-1d84-499c-860a-a60235d45b7d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:47.112267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604f2f4-59b9-41d1-8930-5be565751ca6 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:47.121631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 516/1018] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:47.122769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604f2f4-59b9-41d1-8930-5be565751ca6 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:47.141008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:47.211335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:47.215809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e4c85-1d84-499c-860a-a60235d45b7d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:47.228708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604f2f4-59b9-41d1-8930-5be565751ca6 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:47.251515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc1884-a910-40cb-9547-1b31c87af230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:47.252308 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc1884-a910-40cb-9547-1b31c87af230 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:47.252508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc1884-a910-40cb-9547-1b31c87af230 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:47.252724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc1884-a910-40cb-9547-1b31c87af230 None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:47.253448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc1884-a910-40cb-9547-1b31c87af230 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:47.253756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc1884-a910-40cb-9547-1b31c87af230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:47.255115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc1884-a910-40cb-9547-1b31c87af230 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:47.260917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 503/1019] 10.222.0.81 () {56 vars in 1112 bytes} [Thu Jun 16 18:38:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:47.266364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc1884-a910-40cb-9547-1b31c87af230 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:47.266883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c621d0-b6fd-49b6-a1f0-78f3a3d89ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:47.267549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c621d0-b6fd-49b6-a1f0-78f3a3d89ade None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:47.267835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c621d0-b6fd-49b6-a1f0-78f3a3d89ade None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:47.268113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c621d0-b6fd-49b6-a1f0-78f3a3d89ade None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:47.268818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c621d0-b6fd-49b6-a1f0-78f3a3d89ade None admin] RBAC: Adding query filter params (type=image) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:47.269129 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c621d0-b6fd-49b6-a1f0-78f3a3d89ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:47.270524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c621d0-b6fd-49b6-a1f0-78f3a3d89ade None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:47.275549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 517/1020] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:47.281278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c621d0-b6fd-49b6-a1f0-78f3a3d89ade None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:47.361465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:47.374206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc1884-a910-40cb-9547-1b31c87af230 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:47.395887 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c621d0-b6fd-49b6-a1f0-78f3a3d89ade None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:47.406813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5faf43-e350-42bc-adf2-44d9b42da047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:47.407509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5faf43-e350-42bc-adf2-44d9b42da047 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:47.407795 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5faf43-e350-42bc-adf2-44d9b42da047 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:47.408071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5faf43-e350-42bc-adf2-44d9b42da047 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:47.408798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5faf43-e350-42bc-adf2-44d9b42da047 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:47.410206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5faf43-e350-42bc-adf2-44d9b42da047 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:47.430945 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7ae88-bc01-4e19-b0a1-90e4cabcd736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:47.435141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7ae88-bc01-4e19-b0a1-90e4cabcd736 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:47.435141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7ae88-bc01-4e19-b0a1-90e4cabcd736 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:47.435141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7ae88-bc01-4e19-b0a1-90e4cabcd736 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:47.435141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7ae88-bc01-4e19-b0a1-90e4cabcd736 None admin] RBAC: Adding query filter params (interface=public, service_id=e0f2900df99d44c691db25cc0b99a705, region_id=RegionOne) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:47.435141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7ae88-bc01-4e19-b0a1-90e4cabcd736 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:47.435141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7ae88-bc01-4e19-b0a1-90e4cabcd736 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:47.440254 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 518/1021] 10.222.0.81 () {56 vars in 1251 bytes} [Thu Jun 16 18:38:47 2022] GET /identity/v3/endpoints?service_id=e0f2900df99d44c691db25cc0b99a705&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:47.445766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7ae88-bc01-4e19-b0a1-90e4cabcd736 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:47.457266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:47.499975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 504/1022] 10.222.0.81 () {60 vars in 1144 bytes} [Thu Jun 16 18:38:47 2022] POST /identity/v3/endpoints => generated 334 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:38:47.567081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0f7a7-d9c8-487e-92a5-ac8655b2ef87 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:47.567372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0f7a7-d9c8-487e-92a5-ac8655b2ef87 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:47.567649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0f7a7-d9c8-487e-92a5-ac8655b2ef87 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:47.568193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7ae88-bc01-4e19-b0a1-90e4cabcd736 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:47.569255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 505/1023] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:47.574893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698cf73-909f-4943-9c62-2252c787458c None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:47.575196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698cf73-909f-4943-9c62-2252c787458c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:47.575457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698cf73-909f-4943-9c62-2252c787458c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:47.593094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1698cf73-909f-4943-9c62-2252c787458c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:47.594374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1698cf73-909f-4943-9c62-2252c787458c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:47.616697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e6b01-486d-44ed-a50a-303b8975955d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:47.617456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e6b01-486d-44ed-a50a-303b8975955d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:47.617743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e6b01-486d-44ed-a50a-303b8975955d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:47.618019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e6b01-486d-44ed-a50a-303b8975955d None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:47.618711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e6b01-486d-44ed-a50a-303b8975955d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:47.620115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e6b01-486d-44ed-a50a-303b8975955d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:47.637263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 519/1024] 10.222.0.81 () {56 vars in 1087 bytes} [Thu Jun 16 18:38:47 2022] GET /identity/v3/services => generated 1957 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:38:47.680967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 506/1025] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:47.706791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e6b01-486d-44ed-a50a-303b8975955d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:47.785539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:47.819267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e6b01-486d-44ed-a50a-303b8975955d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:47.857694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf19363-a558-4f5f-bef2-e9ef4ff1a1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:47.858487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf19363-a558-4f5f-bef2-e9ef4ff1a1c8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:47.858768 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf19363-a558-4f5f-bef2-e9ef4ff1a1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:47.859041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf19363-a558-4f5f-bef2-e9ef4ff1a1c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:47.860180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf19363-a558-4f5f-bef2-e9ef4ff1a1c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:47.861656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf19363-a558-4f5f-bef2-e9ef4ff1a1c8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:47.863407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 520/1026] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:47.869274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698cf73-909f-4943-9c62-2252c787458c None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:47.948891 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:47.969913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698cf73-909f-4943-9c62-2252c787458c None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:48.005094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c42af-7f11-4784-9181-b502aec279f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:48.005788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c42af-7f11-4784-9181-b502aec279f8 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:48.006071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c42af-7f11-4784-9181-b502aec279f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:48.006340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c42af-7f11-4784-9181-b502aec279f8 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:48.007037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c42af-7f11-4784-9181-b502aec279f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:48.008462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c42af-7f11-4784-9181-b502aec279f8 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:48.083612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 507/1027] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:38:47 2022] POST /identity/v3/projects => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:49.752236 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6c4f-68d5-42f1-9fdb-aeaa8db53afe None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:49.752729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6c4f-68d5-42f1-9fdb-aeaa8db53afe None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:49.752729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6c4f-68d5-42f1-9fdb-aeaa8db53afe None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:49.757278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 521/1028] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:49.759295 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cd4ed-d188-4b24-89ef-13905aa4671b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:49.759690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cd4ed-d188-4b24-89ef-13905aa4671b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:49.759851 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31cd4ed-d188-4b24-89ef-13905aa4671b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:49.776813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d31cd4ed-d188-4b24-89ef-13905aa4671b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:49.778022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d31cd4ed-d188-4b24-89ef-13905aa4671b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:49.790900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d386400-85b6-4637-a7d0-24f9632e19d2 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:49.791220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d386400-85b6-4637-a7d0-24f9632e19d2 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:49.791460 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d386400-85b6-4637-a7d0-24f9632e19d2 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:49.792939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 522/1029] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:49.797732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418ba5e-a903-42bb-acb3-55264b80b64f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:49.798020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418ba5e-a903-42bb-acb3-55264b80b64f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:49.798295 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418ba5e-a903-42bb-acb3-55264b80b64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:49.816085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9418ba5e-a903-42bb-acb3-55264b80b64f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:49.817286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9418ba5e-a903-42bb-acb3-55264b80b64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:49.856038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 508/1030] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:49.879794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31cd4ed-d188-4b24-89ef-13905aa4671b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:49.888329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 523/1031] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:49.911587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418ba5e-a903-42bb-acb3-55264b80b64f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:49.937771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:49.978103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31cd4ed-d188-4b24-89ef-13905aa4671b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:49.986823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:50.012709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b188429-23a8-4743-9144-bec4a547f212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:50.013481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b188429-23a8-4743-9144-bec4a547f212 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.013765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b188429-23a8-4743-9144-bec4a547f212 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.014040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b188429-23a8-4743-9144-bec4a547f212 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.020105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418ba5e-a903-42bb-acb3-55264b80b64f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:50.025219 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b188429-23a8-4743-9144-bec4a547f212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-5b188429-23a8-4743-9144-bec4a547f212 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:50.033424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:50.034347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:50.035178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 509/1032] 10.222.0.81 () {56 vars in 1120 bytes} [Thu Jun 16 18:38:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:50.041155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b188429-23a8-4743-9144-bec4a547f212 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:50.061628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e3dbd-2b42-4901-8a35-bd9679081023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:50.062426 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e3dbd-2b42-4901-8a35-bd9679081023 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.062711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e3dbd-2b42-4901-8a35-bd9679081023 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.062981 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e3dbd-2b42-4901-8a35-bd9679081023 None admin] PATH_INFO: `/v3/services/image` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.063622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e3dbd-2b42-4901-8a35-bd9679081023 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:50.064992 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e3dbd-2b42-4901-8a35-bd9679081023 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-848e3dbd-2b42-4901-8a35-bd9679081023 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:50.076998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:50.078051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:50.079255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 524/1033] 10.222.0.81 () {56 vars in 1105 bytes} [Thu Jun 16 18:38:49 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:50.086116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e3dbd-2b42-4901-8a35-bd9679081023 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:50.156842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b188429-23a8-4743-9144-bec4a547f212 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:50.182316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:50.192583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e3dbd-2b42-4901-8a35-bd9679081023 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:50.199255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0385d3-566a-459b-9e67-cc83aa34bd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:50.200268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0385d3-566a-459b-9e67-cc83aa34bd15 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.200771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0385d3-566a-459b-9e67-cc83aa34bd15 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.201047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0385d3-566a-459b-9e67-cc83aa34bd15 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.201868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0385d3-566a-459b-9e67-cc83aa34bd15 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:50.202183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0385d3-566a-459b-9e67-cc83aa34bd15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:50.203738 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0385d3-566a-459b-9e67-cc83aa34bd15 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:50.219166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 510/1034] 10.222.0.81 () {56 vars in 1115 bytes} [Thu Jun 16 18:38:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:50.220204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0385d3-566a-459b-9e67-cc83aa34bd15 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:50.222918 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:50.228380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9a9eb-3a99-4f28-b4fd-f767dc9e1247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:50.229047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a9eb-3a99-4f28-b4fd-f767dc9e1247 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.229353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a9eb-3a99-4f28-b4fd-f767dc9e1247 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.232862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a9eb-3a99-4f28-b4fd-f767dc9e1247 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.232862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9a9eb-3a99-4f28-b4fd-f767dc9e1247 None admin] RBAC: Adding query filter params (name=image) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:50.232862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9a9eb-3a99-4f28-b4fd-f767dc9e1247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:50.232862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9a9eb-3a99-4f28-b4fd-f767dc9e1247 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:50.237513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 525/1035] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:50.243299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9a9eb-3a99-4f28-b4fd-f767dc9e1247 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:50.324092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0385d3-566a-459b-9e67-cc83aa34bd15 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:50.340911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9a9eb-3a99-4f28-b4fd-f767dc9e1247 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:50.357477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947dd595-4001-49b2-bef5-b5435f3230ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:50.358260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dd595-4001-49b2-bef5-b5435f3230ae None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.358538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dd595-4001-49b2-bef5-b5435f3230ae None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.358806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dd595-4001-49b2-bef5-b5435f3230ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.373905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947dd595-4001-49b2-bef5-b5435f3230ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:50.375334 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947dd595-4001-49b2-bef5-b5435f3230ae None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:50.377124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3484e5-768a-4431-b12f-f53df8408501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:50.377829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3484e5-768a-4431-b12f-f53df8408501 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.378116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3484e5-768a-4431-b12f-f53df8408501 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.378393 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3484e5-768a-4431-b12f-f53df8408501 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.379096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3484e5-768a-4431-b12f-f53df8408501 None admin] RBAC: Adding query filter params (type=image) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:50.379409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3484e5-768a-4431-b12f-f53df8408501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:50.380796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3484e5-768a-4431-b12f-f53df8408501 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:50.385590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 526/1036] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:50.386783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c2316-832c-4e54-9812-44f570504c8d None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.387073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c2316-832c-4e54-9812-44f570504c8d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.387350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c2316-832c-4e54-9812-44f570504c8d None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.388746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 527/1037] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-947dd595-4001-49b2-bef5-b5435f3230ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:50.392401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:50.393437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:50.394329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:50.394329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:50.394329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:50.394329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:50.394550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c2316-832c-4e54-9812-44f570504c8d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:50.394628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 511/1038] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:50.395370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a97725-32b4-49c0-94b0-4ca9be69f952 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.395656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a97725-32b4-49c0-94b0-4ca9be69f952 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.395932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a97725-32b4-49c0-94b0-4ca9be69f952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.411881 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-82a97725-32b4-49c0-94b0-4ca9be69f952 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:50.413637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-82a97725-32b4-49c0-94b0-4ca9be69f952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:50.439861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:50.453286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:50.479580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 512/1039] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:50.480390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a97725-32b4-49c0-94b0-4ca9be69f952 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:50.494366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c2316-832c-4e54-9812-44f570504c8d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:50.528636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c1eb8-8268-4152-a8e9-2081c4865e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:50.529323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1eb8-8268-4152-a8e9-2081c4865e9e None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.529618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1eb8-8268-4152-a8e9-2081c4865e9e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.529894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c1eb8-8268-4152-a8e9-2081c4865e9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.530620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c1eb8-8268-4152-a8e9-2081c4865e9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:50.531976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c1eb8-8268-4152-a8e9-2081c4865e9e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:50.578521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a97725-32b4-49c0-94b0-4ca9be69f952 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:50.612126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08708e-8123-4924-a324-6d5d33029f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:50.612775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08708e-8123-4924-a324-6d5d33029f32 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.613056 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08708e-8123-4924-a324-6d5d33029f32 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.613359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08708e-8123-4924-a324-6d5d33029f32 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.614064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08708e-8123-4924-a324-6d5d33029f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:50.614369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08708e-8123-4924-a324-6d5d33029f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:50.615761 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a08708e-8123-4924-a324-6d5d33029f32 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:50.626610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 513/1040] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:50.643103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 528/1041] 10.222.0.81 () {60 vars in 1144 bytes} [Thu Jun 16 18:38:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:38:50.643310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08708e-8123-4924-a324-6d5d33029f32 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:50.643419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c1eb8-8268-4152-a8e9-2081c4865e9e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:50.662602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:50.816938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a08708e-8123-4924-a324-6d5d33029f32 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:50.820794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c1eb8-8268-4152-a8e9-2081c4865e9e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:50.835191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:50.857228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173118d5-bdfe-464a-b5f6-f1ce32cf2bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:50.857486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54f321-7e8b-429c-ad1b-89e513d78204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:50.858090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173118d5-bdfe-464a-b5f6-f1ce32cf2bcc None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.858263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54f321-7e8b-429c-ad1b-89e513d78204 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:50.858359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173118d5-bdfe-464a-b5f6-f1ce32cf2bcc None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.858508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54f321-7e8b-429c-ad1b-89e513d78204 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:50.858668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173118d5-bdfe-464a-b5f6-f1ce32cf2bcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.858758 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54f321-7e8b-429c-ad1b-89e513d78204 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:50.859799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173118d5-bdfe-464a-b5f6-f1ce32cf2bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:50.861983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173118d5-bdfe-464a-b5f6-f1ce32cf2bcc None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:50.863781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 514/1042] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:50.866365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54f321-7e8b-429c-ad1b-89e513d78204 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:50.867796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54f321-7e8b-429c-ad1b-89e513d78204 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:50.869399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173118d5-bdfe-464a-b5f6-f1ce32cf2bcc None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-8e54f321-7e8b-429c-ad1b-89e513d78204 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:50.883843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:50.888675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:50.889816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:50.889816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:50.889816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:50.889816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:50.889816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:50.889816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:50.889816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:50.889816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:50.889816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:50.889816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:50.889816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 529/1043] 10.222.0.81 () {56 vars in 1093 bytes} [Thu Jun 16 18:38:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:50.894175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54f321-7e8b-429c-ad1b-89e513d78204 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:50.967447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:50.978669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173118d5-bdfe-464a-b5f6-f1ce32cf2bcc None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:51.000964 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54f321-7e8b-429c-ad1b-89e513d78204 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:51.015144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f32087-1968-4f7d-ac5f-0a75869e498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:51.016022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f32087-1968-4f7d-ac5f-0a75869e498c None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:51.016309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f32087-1968-4f7d-ac5f-0a75869e498c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:51.016585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f32087-1968-4f7d-ac5f-0a75869e498c None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:51.017308 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f32087-1968-4f7d-ac5f-0a75869e498c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:51.018722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f32087-1968-4f7d-ac5f-0a75869e498c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:51.036967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f45a44-08ac-4686-91c9-05ffafef3325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:51.037681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f45a44-08ac-4686-91c9-05ffafef3325 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:51.037956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f45a44-08ac-4686-91c9-05ffafef3325 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:51.042091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f45a44-08ac-4686-91c9-05ffafef3325 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:51.042091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f45a44-08ac-4686-91c9-05ffafef3325 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:51.042091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f45a44-08ac-4686-91c9-05ffafef3325 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:51.042091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f45a44-08ac-4686-91c9-05ffafef3325 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:51.058130 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 530/1044] 10.222.0.81 () {56 vars in 1133 bytes} [Thu Jun 16 18:38:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:51.063702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f45a44-08ac-4686-91c9-05ffafef3325 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:51.093306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:51.101269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 515/1045] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:38:50 2022] POST /identity/v3/users => generated 285 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:38:51.174914 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f45a44-08ac-4686-91c9-05ffafef3325 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:51.211063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59ce81-0937-4903-b5c1-087aedc41ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:51.211721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59ce81-0937-4903-b5c1-087aedc41ea2 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:51.212001 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59ce81-0937-4903-b5c1-087aedc41ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:51.212270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59ce81-0937-4903-b5c1-087aedc41ea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:51.283224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59ce81-0937-4903-b5c1-087aedc41ea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:51.284658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59ce81-0937-4903-b5c1-087aedc41ea2 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:51.314801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:51.390984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 531/1046] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 18:38:51.397406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f32087-1968-4f7d-ac5f-0a75869e498c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:51.496360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f32087-1968-4f7d-ac5f-0a75869e498c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:51.529336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb30f5-cd64-4577-9ee7-23ea169b7311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:51.530125 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb30f5-cd64-4577-9ee7-23ea169b7311 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:51.530404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb30f5-cd64-4577-9ee7-23ea169b7311 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:51.530675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb30f5-cd64-4577-9ee7-23ea169b7311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:51.541494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb30f5-cd64-4577-9ee7-23ea169b7311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:51.542937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb30f5-cd64-4577-9ee7-23ea169b7311 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:51.550578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-76cb30f5-cd64-4577-9ee7-23ea169b7311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:51.560617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:51.564448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:51.565988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:51.565988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:51.565988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:51.565988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:51.565988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 516/1047] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:51.567603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59ce81-0937-4903-b5c1-087aedc41ea2 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:51.679476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59ce81-0937-4903-b5c1-087aedc41ea2 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:51.742361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:51.750266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37beaf4a-b02c-475e-868d-b13069d9db89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:51.750953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beaf4a-b02c-475e-868d-b13069d9db89 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:51.751241 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beaf4a-b02c-475e-868d-b13069d9db89 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:51.751571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beaf4a-b02c-475e-868d-b13069d9db89 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:51.752314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37beaf4a-b02c-475e-868d-b13069d9db89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:51.753135 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37beaf4a-b02c-475e-868d-b13069d9db89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:51.754597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37beaf4a-b02c-475e-868d-b13069d9db89 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:51.772905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 532/1048] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:51.774244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb30f5-cd64-4577-9ee7-23ea169b7311 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:51.902534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb30f5-cd64-4577-9ee7-23ea169b7311 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:51.935727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aca75b-a9f4-46bb-ad49-b7c00adbeddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:51.936444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aca75b-a9f4-46bb-ad49-b7c00adbeddb None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:51.936728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aca75b-a9f4-46bb-ad49-b7c00adbeddb None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:51.937003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aca75b-a9f4-46bb-ad49-b7c00adbeddb None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:51.944380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aca75b-a9f4-46bb-ad49-b7c00adbeddb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:51.945852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aca75b-a9f4-46bb-ad49-b7c00adbeddb None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-91aca75b-a9f4-46bb-ad49-b7c00adbeddb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:51.960604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:51.961545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:51.967213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:51.967213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:51.967213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:51.967213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:51.967213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:51.967213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:51.967213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:51.967213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:51.967213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 517/1049] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:51.969255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37beaf4a-b02c-475e-868d-b13069d9db89 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:52.072204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37beaf4a-b02c-475e-868d-b13069d9db89 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:52.105800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5dbdd-fd29-4085-9f98-e6d2b4af5f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:52.106458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5dbdd-fd29-4085-9f98-e6d2b4af5f38 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:52.106741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5dbdd-fd29-4085-9f98-e6d2b4af5f38 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:52.107015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5dbdd-fd29-4085-9f98-e6d2b4af5f38 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:52.107791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5dbdd-fd29-4085-9f98-e6d2b4af5f38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:52.108101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5dbdd-fd29-4085-9f98-e6d2b4af5f38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:52.109573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5dbdd-fd29-4085-9f98-e6d2b4af5f38 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:52.126364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 533/1050] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:52.128162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aca75b-a9f4-46bb-ad49-b7c00adbeddb None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:52.187438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:52.229187 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aca75b-a9f4-46bb-ad49-b7c00adbeddb None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:52.263603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5251b-3e8f-43ae-980b-c32281528b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:52.264258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5251b-3e8f-43ae-980b-c32281528b94 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:52.264541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5251b-3e8f-43ae-980b-c32281528b94 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:52.264814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5251b-3e8f-43ae-980b-c32281528b94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:52.265660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5251b-3e8f-43ae-980b-c32281528b94 None admin] RBAC: Adding query filter params (role.id=df2b07cfe31149c3990f1f66fdfaed9d, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=251ed026ce8046e98188066bf0bc40b7) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:52.265974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5251b-3e8f-43ae-980b-c32281528b94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:52.267549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5251b-3e8f-43ae-980b-c32281528b94 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:52.281513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 518/1051] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:52 2022] GET /identity/v3/role_assignments?user.id=251ed026ce8046e98188066bf0bc40b7&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=df2b07cfe31149c3990f1f66fdfaed9d => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:52.841400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc25d9-3a6d-48da-b8a6-d93e23282631 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:52.841689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc25d9-3a6d-48da-b8a6-d93e23282631 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:52.841966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc25d9-3a6d-48da-b8a6-d93e23282631 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:52.843328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 534/1052] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:52.848974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3f71f-de70-40e2-82db-bd965ba4c399 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:52.849301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3f71f-de70-40e2-82db-bd965ba4c399 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:52.849586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3f71f-de70-40e2-82db-bd965ba4c399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:52.867636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-58c3f71f-de70-40e2-82db-bd965ba4c399 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:52.868879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-58c3f71f-de70-40e2-82db-bd965ba4c399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:52.938814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 519/1053] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:52.960905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc25d9-3a6d-48da-b8a6-d93e23282631 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:53.012711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:53.065010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc25d9-3a6d-48da-b8a6-d93e23282631 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:53.099134 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a68b81-07e2-4bcf-b0f5-d3881b1b1623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:53.099704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68b81-07e2-4bcf-b0f5-d3881b1b1623 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:53.099988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68b81-07e2-4bcf-b0f5-d3881b1b1623 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:53.100263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a68b81-07e2-4bcf-b0f5-d3881b1b1623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:53.168550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a68b81-07e2-4bcf-b0f5-d3881b1b1623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:53.170021 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a68b81-07e2-4bcf-b0f5-d3881b1b1623 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:53.218480 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:53.268694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 535/1054] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:38:53.274442 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3f71f-de70-40e2-82db-bd965ba4c399 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:53.328412 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:53.387861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3f71f-de70-40e2-82db-bd965ba4c399 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:53.421777 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee65b2-3f8f-4ef9-982b-ab69b89e2898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:53.422575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee65b2-3f8f-4ef9-982b-ab69b89e2898 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:53.422856 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee65b2-3f8f-4ef9-982b-ab69b89e2898 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:53.423129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee65b2-3f8f-4ef9-982b-ab69b89e2898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:53.425642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d3ae9-4149-48e7-a0b1-dc8a2ec8e494 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:53.425938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d3ae9-4149-48e7-a0b1-dc8a2ec8e494 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:53.426230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d3ae9-4149-48e7-a0b1-dc8a2ec8e494 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:53.427607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 536/1055] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:53.432440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61106e-6c53-4a7c-ae0b-b36333638fcc None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:53.432735 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61106e-6c53-4a7c-ae0b-b36333638fcc None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:53.433016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61106e-6c53-4a7c-ae0b-b36333638fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:53.434443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee65b2-3f8f-4ef9-982b-ab69b89e2898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:53.436446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee65b2-3f8f-4ef9-982b-ab69b89e2898 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:53.449812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-5c61106e-6c53-4a7c-ae0b-b36333638fcc None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:53.451010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5c61106e-6c53-4a7c-ae0b-b36333638fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-f4ee65b2-3f8f-4ef9-982b-ab69b89e2898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:53.454813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:53.455888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:53.456913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:53.456913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:53.456913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:53.456913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:53.456913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 520/1056] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:53.462520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee65b2-3f8f-4ef9-982b-ab69b89e2898 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:53.526279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 537/1057] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:53.550181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61106e-6c53-4a7c-ae0b-b36333638fcc None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:53.571834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee65b2-3f8f-4ef9-982b-ab69b89e2898 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:53.580055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:53.611182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6074d-93d4-4fd2-a9e2-d078d918c0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:53.611843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6074d-93d4-4fd2-a9e2-d078d918c0ac None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:53.612123 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6074d-93d4-4fd2-a9e2-d078d918c0ac None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:53.612396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6074d-93d4-4fd2-a9e2-d078d918c0ac None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:53.613163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6074d-93d4-4fd2-a9e2-d078d918c0ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:53.613893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6074d-93d4-4fd2-a9e2-d078d918c0ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:53.616065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6074d-93d4-4fd2-a9e2-d078d918c0ac None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:53.631235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 521/1058] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:53.644152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:53.666141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61106e-6c53-4a7c-ae0b-b36333638fcc None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:53.705335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dec94-f4ab-4f3a-a0ce-e02176a08553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:53.706520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dec94-f4ab-4f3a-a0ce-e02176a08553 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:53.707012 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dec94-f4ab-4f3a-a0ce-e02176a08553 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:53.707512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dec94-f4ab-4f3a-a0ce-e02176a08553 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:53.719730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15dec94-f4ab-4f3a-a0ce-e02176a08553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-f15dec94-f4ab-4f3a-a0ce-e02176a08553 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:53.727831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:53.728807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:53.729721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:53.736349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 538/1059] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:53.737935 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6074d-93d4-4fd2-a9e2-d078d918c0ac None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:53.854805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6074d-93d4-4fd2-a9e2-d078d918c0ac None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:53.891659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c54820-2a23-4c47-b40c-db16382ea35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:53.892267 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c54820-2a23-4c47-b40c-db16382ea35d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:53.892556 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c54820-2a23-4c47-b40c-db16382ea35d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:53.892835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c54820-2a23-4c47-b40c-db16382ea35d None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:53.893627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c54820-2a23-4c47-b40c-db16382ea35d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:53.893940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c54820-2a23-4c47-b40c-db16382ea35d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:53.895314 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c54820-2a23-4c47-b40c-db16382ea35d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:53.905494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 522/1060] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:38:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:53.911563 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dec94-f4ab-4f3a-a0ce-e02176a08553 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:54.014558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dec94-f4ab-4f3a-a0ce-e02176a08553 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:54.047390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed7671-cd2a-428f-8fed-53489502e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:54.048267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed7671-cd2a-428f-8fed-53489502e98c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:54.048545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed7671-cd2a-428f-8fed-53489502e98c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:54.048814 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed7671-cd2a-428f-8fed-53489502e98c None admin] PATH_INFO: `/v3/users/b0ac3e53c6d948a0a42698efdf6b8d7f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:54.059790 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:54.062722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed7671-cd2a-428f-8fed-53489502e98c None admin] RBAC: Authorizing `identity:get_user(user_id=b0ac3e53c6d948a0a42698efdf6b8d7f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:54.064163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed7671-cd2a-428f-8fed-53489502e98c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:54.065827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 539/1061] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:38:53 2022] GET /identity/v3/users/b0ac3e53c6d948a0a42698efdf6b8d7f => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:54.071848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c54820-2a23-4c47-b40c-db16382ea35d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:54.099753 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:54.170396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c54820-2a23-4c47-b40c-db16382ea35d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:54.203211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaccab8-9e6c-471a-8e7b-7c88bef3ccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:54.203892 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaccab8-9e6c-471a-8e7b-7c88bef3ccd6 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:54.204177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaccab8-9e6c-471a-8e7b-7c88bef3ccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:54.204453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaccab8-9e6c-471a-8e7b-7c88bef3ccd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:54.274535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaccab8-9e6c-471a-8e7b-7c88bef3ccd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:54.276754 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaccab8-9e6c-471a-8e7b-7c88bef3ccd6 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:54.303044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:54.380237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 523/1062] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:38:54.386032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed7671-cd2a-428f-8fed-53489502e98c None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:54.410446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad854c-c11f-433a-b6a3-618d04d67147 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:54.410771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad854c-c11f-433a-b6a3-618d04d67147 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:54.411046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad854c-c11f-433a-b6a3-618d04d67147 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:54.412390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 524/1063] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:54.417110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb64f-4495-47f3-9169-9d53637af379 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:54.417438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb64f-4495-47f3-9169-9d53637af379 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:54.417716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb64f-4495-47f3-9169-9d53637af379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:54.433533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-5a7bb64f-4495-47f3-9169-9d53637af379 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:54.434647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5a7bb64f-4495-47f3-9169-9d53637af379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:54.484430 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed7671-cd2a-428f-8fed-53489502e98c None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:54.501328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 525/1064] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:38:54.517476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86897b51-b259-4980-9c1f-8617f0ae8c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:54.518327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897b51-b259-4980-9c1f-8617f0ae8c20 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:54.518610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897b51-b259-4980-9c1f-8617f0ae8c20 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:54.518884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897b51-b259-4980-9c1f-8617f0ae8c20 None admin] PATH_INFO: `/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:54.520043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86897b51-b259-4980-9c1f-8617f0ae8c20 None admin] RBAC: Authorizing `identity:get_project(project_id=09d82e96b9b04be4a0bf2190ef13a1dd)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:54.521520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86897b51-b259-4980-9c1f-8617f0ae8c20 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:54.523180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bb64f-4495-47f3-9169-9d53637af379 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:54.523376 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 540/1065] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:38:54 2022] GET /identity/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:54.528878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86897b51-b259-4980-9c1f-8617f0ae8c20 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:54.581500 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:54.597965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:54.622813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bb64f-4495-47f3-9169-9d53637af379 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:54.633411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86897b51-b259-4980-9c1f-8617f0ae8c20 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:54.655992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727ecc9-3ae3-4202-85f9-818a935bbbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:54.656936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727ecc9-3ae3-4202-85f9-818a935bbbeb None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:54.657206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727ecc9-3ae3-4202-85f9-818a935bbbeb None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:54.657506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727ecc9-3ae3-4202-85f9-818a935bbbeb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:54.666000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf5277-1000-4c78-a5e9-e5220d2517b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:54.666649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf5277-1000-4c78-a5e9-e5220d2517b2 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:54.666930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf5277-1000-4c78-a5e9-e5220d2517b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:54.667203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf5277-1000-4c78-a5e9-e5220d2517b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:54.667460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c727ecc9-3ae3-4202-85f9-818a935bbbeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:54.668575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf5277-1000-4c78-a5e9-e5220d2517b2 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=09d82e96b9b04be4a0bf2190ef13a1dd, user.id=b0ac3e53c6d948a0a42698efdf6b8d7f) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:54.668882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf5277-1000-4c78-a5e9-e5220d2517b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:54.670328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf5277-1000-4c78-a5e9-e5220d2517b2 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:54.675597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 541/1066] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:38:54 2022] GET /identity/v3/role_assignments?user.id=b0ac3e53c6d948a0a42698efdf6b8d7f&scope.project.id=09d82e96b9b04be4a0bf2190ef13a1dd&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-c727ecc9-3ae3-4202-85f9-818a935bbbeb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:54.680943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:54.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:54.683405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 526/1067] 10.222.0.81 () {56 vars in 1120 bytes} [Thu Jun 16 18:38:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:54.684311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf5277-1000-4c78-a5e9-e5220d2517b2 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:54.788015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf5277-1000-4c78-a5e9-e5220d2517b2 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:54.817693 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:54.824074 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1843b95-d306-431a-9114-79c3bab5e593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:54.824302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1843b95-d306-431a-9114-79c3bab5e593 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:54.824589 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1843b95-d306-431a-9114-79c3bab5e593 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:54.824865 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1843b95-d306-431a-9114-79c3bab5e593 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:54.825649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1843b95-d306-431a-9114-79c3bab5e593 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:54.825958 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1843b95-d306-431a-9114-79c3bab5e593 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:54.827326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1843b95-d306-431a-9114-79c3bab5e593 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:54.836519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 542/1068] 10.222.0.81 () {56 vars in 1115 bytes} [Thu Jun 16 18:38:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:54.842167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727ecc9-3ae3-4202-85f9-818a935bbbeb None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:54.939183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727ecc9-3ae3-4202-85f9-818a935bbbeb None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:54.955191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:54.971652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917942c1-6937-4cec-9adf-13c4431c79c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:54.972437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917942c1-6937-4cec-9adf-13c4431c79c8 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:54.972718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917942c1-6937-4cec-9adf-13c4431c79c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:54.972989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917942c1-6937-4cec-9adf-13c4431c79c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:54.984253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917942c1-6937-4cec-9adf-13c4431c79c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:54.985723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917942c1-6937-4cec-9adf-13c4431c79c8 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-917942c1-6937-4cec-9adf-13c4431c79c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:55.002003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:55.003080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:55.004197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:55.004197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:55.004197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:55.004197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:55.005384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 527/1069] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:55.008763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1843b95-d306-431a-9114-79c3bab5e593 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:55.103730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1843b95-d306-431a-9114-79c3bab5e593 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:55.136481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8611e3-4cf6-47ef-8bc4-d51333ac28fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:55.137134 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8611e3-4cf6-47ef-8bc4-d51333ac28fd None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:55.137446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8611e3-4cf6-47ef-8bc4-d51333ac28fd None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:55.137718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8611e3-4cf6-47ef-8bc4-d51333ac28fd None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:55.138419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8611e3-4cf6-47ef-8bc4-d51333ac28fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:55.138721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8611e3-4cf6-47ef-8bc4-d51333ac28fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:55.140065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8611e3-4cf6-47ef-8bc4-d51333ac28fd None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:55.151561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 543/1070] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:55.157051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917942c1-6937-4cec-9adf-13c4431c79c8 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:55.252003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917942c1-6937-4cec-9adf-13c4431c79c8 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:55.283912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ed3b6-5e5f-4676-a228-2ecca8740dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:55.284690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ed3b6-5e5f-4676-a228-2ecca8740dfe None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:55.284967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ed3b6-5e5f-4676-a228-2ecca8740dfe None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:55.285245 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ed3b6-5e5f-4676-a228-2ecca8740dfe None admin] PATH_INFO: `/v3/users/nova` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:55.292460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ed3b6-5e5f-4676-a228-2ecca8740dfe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:55.293917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ed3b6-5e5f-4676-a228-2ecca8740dfe None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-e09ed3b6-5e5f-4676-a228-2ecca8740dfe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:55.307298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:55.308424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:55.309448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:55.309448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:55.309448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:55.309448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:55.309448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:55.309448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:55.309448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:55.309448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:55.309448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:55.309448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:55.313268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 528/1071] 10.222.0.81 () {56 vars in 1093 bytes} [Thu Jun 16 18:38:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:55.314082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8611e3-4cf6-47ef-8bc4-d51333ac28fd None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:55.370456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:55.416618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8611e3-4cf6-47ef-8bc4-d51333ac28fd None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:55.448504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4177826-094d-4054-a091-49c63ad3dacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:55.449171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4177826-094d-4054-a091-49c63ad3dacc None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:55.449486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4177826-094d-4054-a091-49c63ad3dacc None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:55.449763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4177826-094d-4054-a091-49c63ad3dacc None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:55.450583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4177826-094d-4054-a091-49c63ad3dacc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:55.450893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4177826-094d-4054-a091-49c63ad3dacc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:55.452324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4177826-094d-4054-a091-49c63ad3dacc None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:55.470058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 544/1072] 10.222.0.81 () {56 vars in 1133 bytes} [Thu Jun 16 18:38:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:55.475591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ed3b6-5e5f-4676-a228-2ecca8740dfe None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:55.505292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:55.570657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ed3b6-5e5f-4676-a228-2ecca8740dfe None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:55.603447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673d1fe-7b40-4797-8fc2-b7dd9e8e2725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:55.604223 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673d1fe-7b40-4797-8fc2-b7dd9e8e2725 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:55.604503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673d1fe-7b40-4797-8fc2-b7dd9e8e2725 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:55.604773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673d1fe-7b40-4797-8fc2-b7dd9e8e2725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:55.614428 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673d1fe-7b40-4797-8fc2-b7dd9e8e2725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:55.615880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673d1fe-7b40-4797-8fc2-b7dd9e8e2725 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-6673d1fe-7b40-4797-8fc2-b7dd9e8e2725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:55.633537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:55.635027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:55.635910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:55.635910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:55.635910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:55.635910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:55.635910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 529/1073] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:55.640832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4177826-094d-4054-a091-49c63ad3dacc None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:55.773492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4177826-094d-4054-a091-49c63ad3dacc None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:55.784330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:55.808032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b763e0-acd6-4172-be7d-03eb9679cf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:55.808727 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b763e0-acd6-4172-be7d-03eb9679cf65 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:55.809013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b763e0-acd6-4172-be7d-03eb9679cf65 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:55.809320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b763e0-acd6-4172-be7d-03eb9679cf65 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:55.810028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b763e0-acd6-4172-be7d-03eb9679cf65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:55.810336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b763e0-acd6-4172-be7d-03eb9679cf65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:55.811702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b763e0-acd6-4172-be7d-03eb9679cf65 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:55.822785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 545/1074] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:55.828595 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673d1fe-7b40-4797-8fc2-b7dd9e8e2725 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:55.906769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:55.925251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673d1fe-7b40-4797-8fc2-b7dd9e8e2725 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:55.957828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75aa6b5-1071-45d4-9221-98f98499d27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:55.958632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aa6b5-1071-45d4-9221-98f98499d27a None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:55.958922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aa6b5-1071-45d4-9221-98f98499d27a None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:55.959198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aa6b5-1071-45d4-9221-98f98499d27a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:55.965901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75aa6b5-1071-45d4-9221-98f98499d27a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:55.967322 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75aa6b5-1071-45d4-9221-98f98499d27a None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-d75aa6b5-1071-45d4-9221-98f98499d27a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:38:55.981031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:55.981961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:55.982953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:38:55.982953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:38:55.982953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:38:55.982953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:38:55.982953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:38:55.982953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:38:55.982953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:38:55.982953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:55.982953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 530/1075] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:55.987934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b763e0-acd6-4172-be7d-03eb9679cf65 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:56.084352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b763e0-acd6-4172-be7d-03eb9679cf65 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:56.117582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5750a56-ab2f-482c-9c3f-16025889d52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:56.118231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5750a56-ab2f-482c-9c3f-16025889d52c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:56.118517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5750a56-ab2f-482c-9c3f-16025889d52c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:56.118792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5750a56-ab2f-482c-9c3f-16025889d52c None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:56.119561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5750a56-ab2f-482c-9c3f-16025889d52c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:56.119872 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5750a56-ab2f-482c-9c3f-16025889d52c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:56.121346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5750a56-ab2f-482c-9c3f-16025889d52c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:56.133410 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 546/1076] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:38:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:56.139147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75aa6b5-1071-45d4-9221-98f98499d27a None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:56.234661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75aa6b5-1071-45d4-9221-98f98499d27a None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:56.267445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc46e9-6bd5-402b-b0f7-691467029a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:56.268320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc46e9-6bd5-402b-b0f7-691467029a84 None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:56.268608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc46e9-6bd5-402b-b0f7-691467029a84 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:56.268884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc46e9-6bd5-402b-b0f7-691467029a84 None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/251ed026ce8046e98188066bf0bc40b7/roles/df2b07cfe31149c3990f1f66fdfaed9d` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:56.271094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc46e9-6bd5-402b-b0f7-691467029a84 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=251ed026ce8046e98188066bf0bc40b7, role_id=df2b07cfe31149c3990f1f66fdfaed9d)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:56.272522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc46e9-6bd5-402b-b0f7-691467029a84 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:56.370166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 531/1077] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:38:56 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/251ed026ce8046e98188066bf0bc40b7/roles/df2b07cfe31149c3990f1f66fdfaed9d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:38:56.780853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f190d-1866-4292-8162-208038ac1389 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:56.781063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f190d-1866-4292-8162-208038ac1389 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:56.781397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f190d-1866-4292-8162-208038ac1389 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:56.782772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 547/1078] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:56.788181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e741d1c-0bad-4ef5-bff7-6457d2728009 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:56.788479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e741d1c-0bad-4ef5-bff7-6457d2728009 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:56.788760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e741d1c-0bad-4ef5-bff7-6457d2728009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:56.793696 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:56.805596 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-0e741d1c-0bad-4ef5-bff7-6457d2728009 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:56.806775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0e741d1c-0bad-4ef5-bff7-6457d2728009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:56.893663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 532/1079] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:56.905702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f190d-1866-4292-8162-208038ac1389 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:56.945281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:57.023505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f190d-1866-4292-8162-208038ac1389 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:57.057618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793a010-793d-47c3-8202-99102b7c5af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:57.058422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793a010-793d-47c3-8202-99102b7c5af4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:57.058708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793a010-793d-47c3-8202-99102b7c5af4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:57.058985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793a010-793d-47c3-8202-99102b7c5af4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:57.069456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793a010-793d-47c3-8202-99102b7c5af4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-7793a010-793d-47c3-8202-99102b7c5af4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:57.077049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:57.077999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:38:57.078844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 548/1080] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:38:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:57.085774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e741d1c-0bad-4ef5-bff7-6457d2728009 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:57.187873 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e741d1c-0bad-4ef5-bff7-6457d2728009 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:57.218719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:57.221398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06dc71b-3ede-4307-be86-a09c0c2e5438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:57.222065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dc71b-3ede-4307-be86-a09c0c2e5438 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:57.222350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dc71b-3ede-4307-be86-a09c0c2e5438 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:57.222628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dc71b-3ede-4307-be86-a09c0c2e5438 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:57.223362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06dc71b-3ede-4307-be86-a09c0c2e5438 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:57.223671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06dc71b-3ede-4307-be86-a09c0c2e5438 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:57.225649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06dc71b-3ede-4307-be86-a09c0c2e5438 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:57.233546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 533/1081] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:38:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:57.239212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793a010-793d-47c3-8202-99102b7c5af4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:57.327065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:57.336818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793a010-793d-47c3-8202-99102b7c5af4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:57.371085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61320d91-6698-4a1b-b390-1aa9f139e753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:57.371969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61320d91-6698-4a1b-b390-1aa9f139e753 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:57.372250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61320d91-6698-4a1b-b390-1aa9f139e753 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:57.372520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61320d91-6698-4a1b-b390-1aa9f139e753 None admin] PATH_INFO: `/v3/users/b0ac3e53c6d948a0a42698efdf6b8d7f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:57.373700 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61320d91-6698-4a1b-b390-1aa9f139e753 None admin] RBAC: Authorizing `identity:get_user(user_id=b0ac3e53c6d948a0a42698efdf6b8d7f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:57.375217 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61320d91-6698-4a1b-b390-1aa9f139e753 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:57.376831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 549/1082] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:38:57 2022] GET /identity/v3/users/b0ac3e53c6d948a0a42698efdf6b8d7f => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:38:57.383021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06dc71b-3ede-4307-be86-a09c0c2e5438 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:57.481760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06dc71b-3ede-4307-be86-a09c0c2e5438 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:57.514159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e4f0e-7797-4a13-9edd-4cab6a2fe77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:57.515012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e4f0e-7797-4a13-9edd-4cab6a2fe77e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:57.515295 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e4f0e-7797-4a13-9edd-4cab6a2fe77e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:57.515567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e4f0e-7797-4a13-9edd-4cab6a2fe77e None admin] PATH_INFO: `/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:57.516709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e4f0e-7797-4a13-9edd-4cab6a2fe77e None admin] RBAC: Authorizing `identity:get_project(project_id=09d82e96b9b04be4a0bf2190ef13a1dd)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:57.518159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e4f0e-7797-4a13-9edd-4cab6a2fe77e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:57.519822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 534/1083] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:38:57 2022] GET /identity/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:57.526290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61320d91-6698-4a1b-b390-1aa9f139e753 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:57.622776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61320d91-6698-4a1b-b390-1aa9f139e753 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:57.655900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabebb6-3558-462e-84ba-19c3d98e76b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:57.656780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabebb6-3558-462e-84ba-19c3d98e76b0 None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:57.657058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabebb6-3558-462e-84ba-19c3d98e76b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:57.657355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabebb6-3558-462e-84ba-19c3d98e76b0 None admin] PATH_INFO: `/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd/users/b0ac3e53c6d948a0a42698efdf6b8d7f/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:57.659286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabebb6-3558-462e-84ba-19c3d98e76b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=09d82e96b9b04be4a0bf2190ef13a1dd, user_id=b0ac3e53c6d948a0a42698efdf6b8d7f, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:57.660711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabebb6-3558-462e-84ba-19c3d98e76b0 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:57.779722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 550/1084] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:38:57 2022] PUT /identity/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd/users/b0ac3e53c6d948a0a42698efdf6b8d7f/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:38:58.499444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa44e9-24d7-4f08-a08a-eac7f3a2bdf6 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:58.499984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa44e9-24d7-4f08-a08a-eac7f3a2bdf6 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:58.499984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa44e9-24d7-4f08-a08a-eac7f3a2bdf6 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:58.501509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 535/1085] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:38:58.506992 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c818d38-9a10-471b-b5e6-13bff9162c5c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:58.507368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c818d38-9a10-471b-b5e6-13bff9162c5c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:58.507675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c818d38-9a10-471b-b5e6-13bff9162c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:58.524788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4c818d38-9a10-471b-b5e6-13bff9162c5c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:58.526008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4c818d38-9a10-471b-b5e6-13bff9162c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:58.539831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:58.591718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 551/1086] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:38:58.614993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa44e9-24d7-4f08-a08a-eac7f3a2bdf6 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:58.675402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:58.713917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa44e9-24d7-4f08-a08a-eac7f3a2bdf6 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:58.746366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da34708-0e57-4e8c-a7ef-e48099be57d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:58.747163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da34708-0e57-4e8c-a7ef-e48099be57d4 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:58.747450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da34708-0e57-4e8c-a7ef-e48099be57d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:58.747726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da34708-0e57-4e8c-a7ef-e48099be57d4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:58.758251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da34708-0e57-4e8c-a7ef-e48099be57d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-5da34708-0e57-4e8c-a7ef-e48099be57d4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:38:58.766323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:58.767436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:58.768311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 536/1087] 10.222.0.81 () {56 vars in 1120 bytes} [Thu Jun 16 18:38:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:58.773339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c818d38-9a10-471b-b5e6-13bff9162c5c None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:58.870945 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c818d38-9a10-471b-b5e6-13bff9162c5c None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:58.903256 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a470d-a58d-4a92-8c3d-7a522dbd6a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:58.903920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a470d-a58d-4a92-8c3d-7a522dbd6a90 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:58.904206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a470d-a58d-4a92-8c3d-7a522dbd6a90 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:58.904485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a470d-a58d-4a92-8c3d-7a522dbd6a90 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:58.905221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a470d-a58d-4a92-8c3d-7a522dbd6a90 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:58.905558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a470d-a58d-4a92-8c3d-7a522dbd6a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:58.906930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a470d-a58d-4a92-8c3d-7a522dbd6a90 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:58.915274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 552/1088] 10.222.0.81 () {56 vars in 1115 bytes} [Thu Jun 16 18:38:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:58.920890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da34708-0e57-4e8c-a7ef-e48099be57d4 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:59.016661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da34708-0e57-4e8c-a7ef-e48099be57d4 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:59.038194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:59.049504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c9700-e043-4b09-858c-d453c302d101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:59.050282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9700-e043-4b09-858c-d453c302d101 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:59.050563 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9700-e043-4b09-858c-d453c302d101 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:59.050833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c9700-e043-4b09-858c-d453c302d101 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:59.060802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c9700-e043-4b09-858c-d453c302d101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:59.062255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c9700-e043-4b09-858c-d453c302d101 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-971c9700-e043-4b09-858c-d453c302d101 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:38:59.078752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:59.079951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:59.080835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:38:59.080835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:38:59.080835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:38:59.080835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:59.080835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 537/1089] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:38:59.085888 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a470d-a58d-4a92-8c3d-7a522dbd6a90 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:59.098726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:59.181500 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a470d-a58d-4a92-8c3d-7a522dbd6a90 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:59.213400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581f718-c57d-44fc-872d-1f88b925d882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:59.214047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581f718-c57d-44fc-872d-1f88b925d882 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:59.214332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581f718-c57d-44fc-872d-1f88b925d882 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:59.214605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581f718-c57d-44fc-872d-1f88b925d882 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:59.215315 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581f718-c57d-44fc-872d-1f88b925d882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:59.215622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581f718-c57d-44fc-872d-1f88b925d882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:59.216980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581f718-c57d-44fc-872d-1f88b925d882 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:59.228698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 553/1090] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:59.234136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c9700-e043-4b09-858c-d453c302d101 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:59.329436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c9700-e043-4b09-858c-d453c302d101 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:59.361940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cf017-7cab-4361-bace-28fa9ebc864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:59.362729 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cf017-7cab-4361-bace-28fa9ebc864b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:59.363013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cf017-7cab-4361-bace-28fa9ebc864b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:59.363289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cf017-7cab-4361-bace-28fa9ebc864b None admin] PATH_INFO: `/v3/users/nova` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:59.371546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cf017-7cab-4361-bace-28fa9ebc864b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:59.372972 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cf017-7cab-4361-bace-28fa9ebc864b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-a74cf017-7cab-4361-bace-28fa9ebc864b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:38:59.386543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:38:59.387656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:38:59.388758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:38:59.388758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:38:59.388758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:38:59.388758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:38:59.388758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:38:59.388758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:38:59.388758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:38:59.388758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:38:59.388758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 18:38:59.388758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:38:59.388758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 538/1091] 10.222.0.81 () {56 vars in 1093 bytes} [Thu Jun 16 18:38:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:38:59.393957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581f718-c57d-44fc-872d-1f88b925d882 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:59.460931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:59.491259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581f718-c57d-44fc-872d-1f88b925d882 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:59.524664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104cfdf-6590-4d33-a717-570022f42902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:59.525359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104cfdf-6590-4d33-a717-570022f42902 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:59.525647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104cfdf-6590-4d33-a717-570022f42902 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:59.525926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104cfdf-6590-4d33-a717-570022f42902 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:59.526748 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104cfdf-6590-4d33-a717-570022f42902 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:38:59.527061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104cfdf-6590-4d33-a717-570022f42902 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:59.528489 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104cfdf-6590-4d33-a717-570022f42902 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:59.545173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 554/1092] 10.222.0.81 () {56 vars in 1133 bytes} [Thu Jun 16 18:38:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:38:59.551071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cf017-7cab-4361-bace-28fa9ebc864b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:59.585107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:59.648966 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cf017-7cab-4361-bace-28fa9ebc864b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:59.681118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fa3b4-75f8-4b68-abd9-689526001d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:59.681772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fa3b4-75f8-4b68-abd9-689526001d19 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:59.682060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fa3b4-75f8-4b68-abd9-689526001d19 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:59.682348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fa3b4-75f8-4b68-abd9-689526001d19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:59.748156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fa3b4-75f8-4b68-abd9-689526001d19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:38:59.750090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fa3b4-75f8-4b68-abd9-689526001d19 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:38:59.781567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:59.847866 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 539/1093] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:38:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:38:59.854122 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104cfdf-6590-4d33-a717-570022f42902 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:38:59.897853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d49313-bf20-4205-a20b-c670e69c6906 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:59.898145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d49313-bf20-4205-a20b-c670e69c6906 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:59.898423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d49313-bf20-4205-a20b-c670e69c6906 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:59.899815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 540/1094] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:38:59.904612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602583b-8445-4dfc-a648-bdcaf06410a4 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:59.904904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602583b-8445-4dfc-a648-bdcaf06410a4 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:59.905186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602583b-8445-4dfc-a648-bdcaf06410a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:59.922006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6602583b-8445-4dfc-a648-bdcaf06410a4 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:38:59.923194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6602583b-8445-4dfc-a648-bdcaf06410a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:38:59.953681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104cfdf-6590-4d33-a717-570022f42902 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:38:59.980113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:38:59.988338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e64c9b-69c9-46fd-885d-23dfe6eb813a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:38:59.989117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e64c9b-69c9-46fd-885d-23dfe6eb813a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:38:59.995600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e64c9b-69c9-46fd-885d-23dfe6eb813a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:38:59.995600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e64c9b-69c9-46fd-885d-23dfe6eb813a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:38:59.995879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 541/1095] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:38:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:39:00.000067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e64c9b-69c9-46fd-885d-23dfe6eb813a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:00.001482 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e64c9b-69c9-46fd-885d-23dfe6eb813a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:00.014822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602583b-8445-4dfc-a648-bdcaf06410a4 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-94e64c9b-69c9-46fd-885d-23dfe6eb813a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:39:00.018367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:39:00.019299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:39:00.020326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:39:00.020326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:39:00.020326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:39:00.020326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:39:00.020326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 555/1096] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:39:00.025339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e64c9b-69c9-46fd-885d-23dfe6eb813a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:00.075775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:00.112504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602583b-8445-4dfc-a648-bdcaf06410a4 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:00.121804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e64c9b-69c9-46fd-885d-23dfe6eb813a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:00.145696 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38367d1e-6464-4ecf-83c4-4b078f3c3147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:00.146484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38367d1e-6464-4ecf-83c4-4b078f3c3147 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:00.146774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38367d1e-6464-4ecf-83c4-4b078f3c3147 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:00.147077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38367d1e-6464-4ecf-83c4-4b078f3c3147 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:00.154672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9b4f5-71de-4d40-98e0-f086e1f55b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:00.155346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9b4f5-71de-4d40-98e0-f086e1f55b8a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:00.155629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9b4f5-71de-4d40-98e0-f086e1f55b8a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:00.155905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9b4f5-71de-4d40-98e0-f086e1f55b8a None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:00.156612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9b4f5-71de-4d40-98e0-f086e1f55b8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:00.156920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9b4f5-71de-4d40-98e0-f086e1f55b8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:00.157257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38367d1e-6464-4ecf-83c4-4b078f3c3147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:39:00.158306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9b4f5-71de-4d40-98e0-f086e1f55b8a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-38367d1e-6464-4ecf-83c4-4b078f3c3147 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:39:00.164792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:39:00.165940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:39:00.166888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 542/1097] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:39:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:39:00.175907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 556/1098] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:00.176046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38367d1e-6464-4ecf-83c4-4b078f3c3147 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:00.176870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9b4f5-71de-4d40-98e0-f086e1f55b8a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:00.217380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:00.271505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38367d1e-6464-4ecf-83c4-4b078f3c3147 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:00.273579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9b4f5-71de-4d40-98e0-f086e1f55b8a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:00.293734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:00.304573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4493f9f-a3e3-402b-842b-e4753ac53662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:00.305288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4493f9f-a3e3-402b-842b-e4753ac53662 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:00.305546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4493f9f-a3e3-402b-842b-e4753ac53662 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:00.305824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4493f9f-a3e3-402b-842b-e4753ac53662 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:00.306552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4493f9f-a3e3-402b-842b-e4753ac53662 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:00.306809 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bebdd-0404-48e3-9109-8e57f247107e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:00.306929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4493f9f-a3e3-402b-842b-e4753ac53662 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:00.307599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bebdd-0404-48e3-9109-8e57f247107e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:00.307888 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bebdd-0404-48e3-9109-8e57f247107e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:00.308167 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bebdd-0404-48e3-9109-8e57f247107e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:00.308253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4493f9f-a3e3-402b-842b-e4753ac53662 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:00.314819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bebdd-0404-48e3-9109-8e57f247107e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:00.316292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bebdd-0404-48e3-9109-8e57f247107e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:00.317158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 543/1099] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:39:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:39:00.322879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4493f9f-a3e3-402b-842b-e4753ac53662 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-5e8bebdd-0404-48e3-9109-8e57f247107e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:39:00.331029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:39:00.331973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:39:00.332860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:39:00.332860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:39:00.332860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:39:00.332860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:39:00.332860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:39:00.332860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:39:00.332860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:39:00.332860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:39:00.332860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 557/1100] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:39:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:39:00.337875 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bebdd-0404-48e3-9109-8e57f247107e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:00.424752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4493f9f-a3e3-402b-842b-e4753ac53662 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:00.432900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:00.436482 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bebdd-0404-48e3-9109-8e57f247107e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:00.460212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31b666-1fb2-4b25-a738-ae8ded59cdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:00.461080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31b666-1fb2-4b25-a738-ae8ded59cdbc None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:00.461392 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31b666-1fb2-4b25-a738-ae8ded59cdbc None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:00.461668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31b666-1fb2-4b25-a738-ae8ded59cdbc None admin] PATH_INFO: `/v3/users/b0ac3e53c6d948a0a42698efdf6b8d7f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:00.462858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31b666-1fb2-4b25-a738-ae8ded59cdbc None admin] RBAC: Authorizing `identity:get_user(user_id=b0ac3e53c6d948a0a42698efdf6b8d7f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:00.464294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31b666-1fb2-4b25-a738-ae8ded59cdbc None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:00.466148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 544/1101] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:00 2022] GET /identity/v3/users/b0ac3e53c6d948a0a42698efdf6b8d7f => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:39:00.470038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688dd1b-4ae5-445c-ba1c-d1434d5027c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:00.471461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688dd1b-4ae5-445c-ba1c-d1434d5027c3 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:00.471461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688dd1b-4ae5-445c-ba1c-d1434d5027c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:00.471461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688dd1b-4ae5-445c-ba1c-d1434d5027c3 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:00.471983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688dd1b-4ae5-445c-ba1c-d1434d5027c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:00.472291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688dd1b-4ae5-445c-ba1c-d1434d5027c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:00.472417 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31b666-1fb2-4b25-a738-ae8ded59cdbc None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:00.473745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688dd1b-4ae5-445c-ba1c-d1434d5027c3 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:00.485808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 558/1102] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:39:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:39:00.491588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688dd1b-4ae5-445c-ba1c-d1434d5027c3 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:00.526133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:00.570283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31b666-1fb2-4b25-a738-ae8ded59cdbc None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:00.587377 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688dd1b-4ae5-445c-ba1c-d1434d5027c3 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:00.603412 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393cb9c-d8c8-47cd-abc4-95558ec63bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:00.604044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393cb9c-d8c8-47cd-abc4-95558ec63bcb None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:00.604331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393cb9c-d8c8-47cd-abc4-95558ec63bcb None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:00.604606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393cb9c-d8c8-47cd-abc4-95558ec63bcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:00.620032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781aa2ff-8d16-4b14-99e0-a38969443e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:00.620661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781aa2ff-8d16-4b14-99e0-a38969443e69 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:00.620948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781aa2ff-8d16-4b14-99e0-a38969443e69 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:00.621222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781aa2ff-8d16-4b14-99e0-a38969443e69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:00.622069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781aa2ff-8d16-4b14-99e0-a38969443e69 None admin] RBAC: Adding query filter params (role.id=df2b07cfe31149c3990f1f66fdfaed9d, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=251ed026ce8046e98188066bf0bc40b7) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:00.622368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781aa2ff-8d16-4b14-99e0-a38969443e69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:00.623801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781aa2ff-8d16-4b14-99e0-a38969443e69 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:00.638959 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 559/1103] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:39:00 2022] GET /identity/v3/role_assignments?user.id=251ed026ce8046e98188066bf0bc40b7&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=df2b07cfe31149c3990f1f66fdfaed9d => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:39:00.673677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393cb9c-d8c8-47cd-abc4-95558ec63bcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:00.675147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393cb9c-d8c8-47cd-abc4-95558ec63bcb None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:00.731524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:00.778066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 545/1104] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:39:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:39:00.784061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781aa2ff-8d16-4b14-99e0-a38969443e69 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:00.811611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:00.888407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781aa2ff-8d16-4b14-99e0-a38969443e69 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:00.922107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b0984-7c7b-47ff-bd89-ae6d7b94fd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:00.922968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b0984-7c7b-47ff-bd89-ae6d7b94fd8e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:00.923246 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b0984-7c7b-47ff-bd89-ae6d7b94fd8e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:00.923518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b0984-7c7b-47ff-bd89-ae6d7b94fd8e None admin] PATH_INFO: `/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:00.924671 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b0984-7c7b-47ff-bd89-ae6d7b94fd8e None admin] RBAC: Authorizing `identity:get_project(project_id=09d82e96b9b04be4a0bf2190ef13a1dd)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:00.926176 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b0984-7c7b-47ff-bd89-ae6d7b94fd8e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:00.928015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 560/1105] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:39:00 2022] GET /identity/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:00.933992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393cb9c-d8c8-47cd-abc4-95558ec63bcb None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:01.042575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393cb9c-d8c8-47cd-abc4-95558ec63bcb None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:01.077794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d125cf8-ba80-471c-9a19-7feb593ac16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:01.078441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d125cf8-ba80-471c-9a19-7feb593ac16d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:01.078720 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d125cf8-ba80-471c-9a19-7feb593ac16d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:01.078988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d125cf8-ba80-471c-9a19-7feb593ac16d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:01.079803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d125cf8-ba80-471c-9a19-7feb593ac16d None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=09d82e96b9b04be4a0bf2190ef13a1dd, user.id=b0ac3e53c6d948a0a42698efdf6b8d7f) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:01.080156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d125cf8-ba80-471c-9a19-7feb593ac16d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:01.081738 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d125cf8-ba80-471c-9a19-7feb593ac16d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:01.087706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 546/1106] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:39:00 2022] GET /identity/v3/role_assignments?user.id=b0ac3e53c6d948a0a42698efdf6b8d7f&scope.project.id=09d82e96b9b04be4a0bf2190ef13a1dd&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:03.236470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23497da4-7d75-4ae0-a727-d5e6ed6615d8 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:03.237102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23497da4-7d75-4ae0-a727-d5e6ed6615d8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:03.237102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23497da4-7d75-4ae0-a727-d5e6ed6615d8 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:03.238409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 561/1107] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:03.244119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117fd6-2fd7-4cde-a3a4-b21b993562a1 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:03.244502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117fd6-2fd7-4cde-a3a4-b21b993562a1 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:03.244672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117fd6-2fd7-4cde-a3a4-b21b993562a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:03.261430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-78117fd6-2fd7-4cde-a3a4-b21b993562a1 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:03.262606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-78117fd6-2fd7-4cde-a3a4-b21b993562a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:03.275184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:03.326438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 547/1108] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:03.350436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23497da4-7d75-4ae0-a727-d5e6ed6615d8 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:03.439050 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:03.449027 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23497da4-7d75-4ae0-a727-d5e6ed6615d8 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:03.481665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89374c8c-4a3c-48e2-8945-0a6557634e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:03.482440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89374c8c-4a3c-48e2-8945-0a6557634e52 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:03.482723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89374c8c-4a3c-48e2-8945-0a6557634e52 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:03.482997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89374c8c-4a3c-48e2-8945-0a6557634e52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:03.484156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89374c8c-4a3c-48e2-8945-0a6557634e52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:03.485621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89374c8c-4a3c-48e2-8945-0a6557634e52 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:03.487253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 562/1109] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:39:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:03.493492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78117fd6-2fd7-4cde-a3a4-b21b993562a1 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:03.587260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78117fd6-2fd7-4cde-a3a4-b21b993562a1 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:03.619512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724b690-efc6-43d7-ab41-b0ea63bd3045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:03.620165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724b690-efc6-43d7-ab41-b0ea63bd3045 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:03.620446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724b690-efc6-43d7-ab41-b0ea63bd3045 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:03.620718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724b690-efc6-43d7-ab41-b0ea63bd3045 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:03.621442 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724b690-efc6-43d7-ab41-b0ea63bd3045 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:03.622860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724b690-efc6-43d7-ab41-b0ea63bd3045 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:03.733818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 548/1110] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:39:03 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:05.879447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716187ff-b369-47ba-b58b-4ed201fa5c6d None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:05.879970 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716187ff-b369-47ba-b58b-4ed201fa5c6d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:05.880086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716187ff-b369-47ba-b58b-4ed201fa5c6d None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:05.881384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 563/1111] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:05.886524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdda1c8-10cb-4399-a0a3-452e94af8a0a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:05.886932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdda1c8-10cb-4399-a0a3-452e94af8a0a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:05.887077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdda1c8-10cb-4399-a0a3-452e94af8a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:05.892895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:05.903957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ccdda1c8-10cb-4399-a0a3-452e94af8a0a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:05.907663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ccdda1c8-10cb-4399-a0a3-452e94af8a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:05.969360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 549/1112] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:05.992115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716187ff-b369-47ba-b58b-4ed201fa5c6d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:06.090785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716187ff-b369-47ba-b58b-4ed201fa5c6d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:06.122515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836d776-d8ab-4197-9196-6f371360ef5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:06.123346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836d776-d8ab-4197-9196-6f371360ef5d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:06.123629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836d776-d8ab-4197-9196-6f371360ef5d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:06.123905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836d776-d8ab-4197-9196-6f371360ef5d None admin] PATH_INFO: `/v3/roles/5f24daf2ba2b4f8fab95c87cefb52231` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:06.125380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836d776-d8ab-4197-9196-6f371360ef5d None admin] RBAC: Authorizing `identity:get_role(role_id=5f24daf2ba2b4f8fab95c87cefb52231)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:06.127306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836d776-d8ab-4197-9196-6f371360ef5d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:06.128631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 564/1113] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:05 2022] GET /identity/v3/roles/5f24daf2ba2b4f8fab95c87cefb52231 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:06.134191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdda1c8-10cb-4399-a0a3-452e94af8a0a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:06.228145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdda1c8-10cb-4399-a0a3-452e94af8a0a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:06.260951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356426d-eab7-4ca1-a028-de3857dbeb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:06.261840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356426d-eab7-4ca1-a028-de3857dbeb95 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:06.262122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356426d-eab7-4ca1-a028-de3857dbeb95 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:06.262395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356426d-eab7-4ca1-a028-de3857dbeb95 None admin] PATH_INFO: `/v3/users/312b2cc088c84ac3abfe650ee9e4e1a1` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:06.264223 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:06.275304 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356426d-eab7-4ca1-a028-de3857dbeb95 None admin] RBAC: Authorizing `identity:get_user(user_id=312b2cc088c84ac3abfe650ee9e4e1a1)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:06.276732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356426d-eab7-4ca1-a028-de3857dbeb95 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:06.278367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 550/1114] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:06 2022] GET /identity/v3/users/312b2cc088c84ac3abfe650ee9e4e1a1 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:06.283982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836d776-d8ab-4197-9196-6f371360ef5d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:06.311374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:06.381178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836d776-d8ab-4197-9196-6f371360ef5d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:06.413053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b580cd8-d9ad-4128-a875-db0fc920773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:06.413678 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b580cd8-d9ad-4128-a875-db0fc920773c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:06.413968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b580cd8-d9ad-4128-a875-db0fc920773c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:06.414245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b580cd8-d9ad-4128-a875-db0fc920773c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:06.478920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b580cd8-d9ad-4128-a875-db0fc920773c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:06.480369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b580cd8-d9ad-4128-a875-db0fc920773c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:06.505474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:06.573287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 565/1115] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:39:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:39:06.579752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356426d-eab7-4ca1-a028-de3857dbeb95 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:06.672379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356426d-eab7-4ca1-a028-de3857dbeb95 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:06.704709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f79712-7a10-444d-8b29-f324f7283186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:06.705576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f79712-7a10-444d-8b29-f324f7283186 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:06.705856 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f79712-7a10-444d-8b29-f324f7283186 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:06.706126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f79712-7a10-444d-8b29-f324f7283186 None admin] PATH_INFO: `/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:06.707262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f79712-7a10-444d-8b29-f324f7283186 None admin] RBAC: Authorizing `identity:get_project(project_id=09d82e96b9b04be4a0bf2190ef13a1dd)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:06.708675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f79712-7a10-444d-8b29-f324f7283186 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:06.710324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 551/1116] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:39:06 2022] GET /identity/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:06.716086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b580cd8-d9ad-4128-a875-db0fc920773c None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:06.810269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b580cd8-d9ad-4128-a875-db0fc920773c None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:06.835734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:06.843059 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fef2d-9ca6-4783-ab95-226f20c8f785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:06.843692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fef2d-9ca6-4783-ab95-226f20c8f785 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:06.843997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fef2d-9ca6-4783-ab95-226f20c8f785 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:06.844767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fef2d-9ca6-4783-ab95-226f20c8f785 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:06.845615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2fef2d-9ca6-4783-ab95-226f20c8f785 None admin] RBAC: Adding query filter params (role.id=5f24daf2ba2b4f8fab95c87cefb52231, scope.project.id=09d82e96b9b04be4a0bf2190ef13a1dd, user.id=312b2cc088c84ac3abfe650ee9e4e1a1) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:06.845930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2fef2d-9ca6-4783-ab95-226f20c8f785 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:06.847349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2fef2d-9ca6-4783-ab95-226f20c8f785 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:06.852490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 566/1117] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:39:06 2022] GET /identity/v3/role_assignments?user.id=312b2cc088c84ac3abfe650ee9e4e1a1&scope.project.id=09d82e96b9b04be4a0bf2190ef13a1dd&role.id=5f24daf2ba2b4f8fab95c87cefb52231 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:08.956037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e1738-6cb3-4de9-bfd4-4b90a0e66ad0 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:08.956606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e1738-6cb3-4de9-bfd4-4b90a0e66ad0 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:08.956606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e1738-6cb3-4de9-bfd4-4b90a0e66ad0 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:08.957980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 552/1118] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:08.963053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ac342-5283-4dbf-8f7e-fd1cae69df46 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:08.963471 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ac342-5283-4dbf-8f7e-fd1cae69df46 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:08.963610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ac342-5283-4dbf-8f7e-fd1cae69df46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:08.980291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-637ac342-5283-4dbf-8f7e-fd1cae69df46 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:08.982070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-637ac342-5283-4dbf-8f7e-fd1cae69df46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:09.045230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 567/1119] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:09.067260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e1738-6cb3-4de9-bfd4-4b90a0e66ad0 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:09.124917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:09.163756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e1738-6cb3-4de9-bfd4-4b90a0e66ad0 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:09.196013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73860d14-8fae-4d6a-903f-7af71b8bde1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:09.196847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860d14-8fae-4d6a-903f-7af71b8bde1b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:09.197128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860d14-8fae-4d6a-903f-7af71b8bde1b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:09.197429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860d14-8fae-4d6a-903f-7af71b8bde1b None admin] PATH_INFO: `/v3/roles/5f24daf2ba2b4f8fab95c87cefb52231` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:09.198879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73860d14-8fae-4d6a-903f-7af71b8bde1b None admin] RBAC: Authorizing `identity:get_role(role_id=5f24daf2ba2b4f8fab95c87cefb52231)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:09.200240 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73860d14-8fae-4d6a-903f-7af71b8bde1b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:09.201604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 553/1120] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:09 2022] GET /identity/v3/roles/5f24daf2ba2b4f8fab95c87cefb52231 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:09.209319 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ac342-5283-4dbf-8f7e-fd1cae69df46 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:09.305185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ac342-5283-4dbf-8f7e-fd1cae69df46 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:09.327366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:09.338326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e5c06-fc16-4faf-b220-86034af72531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:09.339754 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5c06-fc16-4faf-b220-86034af72531 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:09.340707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5c06-fc16-4faf-b220-86034af72531 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:09.340707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e5c06-fc16-4faf-b220-86034af72531 None admin] PATH_INFO: `/v3/users/312b2cc088c84ac3abfe650ee9e4e1a1` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:09.341573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e5c06-fc16-4faf-b220-86034af72531 None admin] RBAC: Authorizing `identity:get_user(user_id=312b2cc088c84ac3abfe650ee9e4e1a1)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:09.343022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e5c06-fc16-4faf-b220-86034af72531 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:09.344639 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 568/1121] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:09 2022] GET /identity/v3/users/312b2cc088c84ac3abfe650ee9e4e1a1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:09.350496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73860d14-8fae-4d6a-903f-7af71b8bde1b None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:09.446583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73860d14-8fae-4d6a-903f-7af71b8bde1b None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:09.471808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:09.479055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5484d6-f167-4c38-8f22-ac008f61d7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:09.480403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5484d6-f167-4c38-8f22-ac008f61d7b9 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:09.480682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5484d6-f167-4c38-8f22-ac008f61d7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:09.483497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5484d6-f167-4c38-8f22-ac008f61d7b9 None admin] PATH_INFO: `/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:09.483497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5484d6-f167-4c38-8f22-ac008f61d7b9 None admin] RBAC: Authorizing `identity:get_project(project_id=09d82e96b9b04be4a0bf2190ef13a1dd)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:09.483687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5484d6-f167-4c38-8f22-ac008f61d7b9 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:09.485294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 554/1122] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:39:09 2022] GET /identity/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:09.491235 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e5c06-fc16-4faf-b220-86034af72531 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:09.587816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e5c06-fc16-4faf-b220-86034af72531 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:09.620886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd5c49-85bc-40d2-a143-82be081f77d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:09.621790 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd5c49-85bc-40d2-a143-82be081f77d5 None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:09.622073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd5c49-85bc-40d2-a143-82be081f77d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:09.622344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd5c49-85bc-40d2-a143-82be081f77d5 None admin] PATH_INFO: `/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd/users/312b2cc088c84ac3abfe650ee9e4e1a1/roles/5f24daf2ba2b4f8fab95c87cefb52231` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:09.624495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd5c49-85bc-40d2-a143-82be081f77d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=09d82e96b9b04be4a0bf2190ef13a1dd, user_id=312b2cc088c84ac3abfe650ee9e4e1a1, role_id=5f24daf2ba2b4f8fab95c87cefb52231)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:09.625946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd5c49-85bc-40d2-a143-82be081f77d5 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:09.681966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 569/1123] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:39:09 2022] PUT /identity/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd/users/312b2cc088c84ac3abfe650ee9e4e1a1/roles/5f24daf2ba2b4f8fab95c87cefb52231 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:11.782114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6d56c-a066-4176-a5be-667fdcc153f3 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:11.782629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6d56c-a066-4176-a5be-667fdcc153f3 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:11.782629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb6d56c-a066-4176-a5be-667fdcc153f3 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:11.783934 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 555/1124] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:11.789161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db28ea-274f-4134-9f8e-b2aef1813627 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:11.789636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db28ea-274f-4134-9f8e-b2aef1813627 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:11.789743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db28ea-274f-4134-9f8e-b2aef1813627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:11.805915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-51db28ea-274f-4134-9f8e-b2aef1813627 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:11.807065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-51db28ea-274f-4134-9f8e-b2aef1813627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:11.850734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:11.870631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 570/1125] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:39:11.895124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6d56c-a066-4176-a5be-667fdcc153f3 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:11.992671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb6d56c-a066-4176-a5be-667fdcc153f3 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:12.024556 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6f15d-e13f-4f8f-bfbb-21762a4f93e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:12.025432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6f15d-e13f-4f8f-bfbb-21762a4f93e8 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:12.025724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6f15d-e13f-4f8f-bfbb-21762a4f93e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:12.025998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6f15d-e13f-4f8f-bfbb-21762a4f93e8 None admin] PATH_INFO: `/v3/roles/5f24daf2ba2b4f8fab95c87cefb52231` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:12.027457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6f15d-e13f-4f8f-bfbb-21762a4f93e8 None admin] RBAC: Authorizing `identity:get_role(role_id=5f24daf2ba2b4f8fab95c87cefb52231)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:12.028824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6f15d-e13f-4f8f-bfbb-21762a4f93e8 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:12.030196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 556/1126] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:11 2022] GET /identity/v3/roles/5f24daf2ba2b4f8fab95c87cefb52231 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:12.035996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db28ea-274f-4134-9f8e-b2aef1813627 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:12.130743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db28ea-274f-4134-9f8e-b2aef1813627 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:12.163279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfaca6-ed6a-4489-9882-d672232d598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:12.164124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfaca6-ed6a-4489-9882-d672232d598a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:12.164408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfaca6-ed6a-4489-9882-d672232d598a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:12.164679 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfaca6-ed6a-4489-9882-d672232d598a None admin] PATH_INFO: `/v3/users/312b2cc088c84ac3abfe650ee9e4e1a1` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:12.165851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfaca6-ed6a-4489-9882-d672232d598a None admin] RBAC: Authorizing `identity:get_user(user_id=312b2cc088c84ac3abfe650ee9e4e1a1)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:12.167259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfaca6-ed6a-4489-9882-d672232d598a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:12.168839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 571/1127] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:12 2022] GET /identity/v3/users/312b2cc088c84ac3abfe650ee9e4e1a1 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:12.174390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6f15d-e13f-4f8f-bfbb-21762a4f93e8 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:12.233898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:12.269155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6f15d-e13f-4f8f-bfbb-21762a4f93e8 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:12.301595 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76ceb3-3daf-492e-a927-06c395f7cd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:12.302179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76ceb3-3daf-492e-a927-06c395f7cd42 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:12.302496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76ceb3-3daf-492e-a927-06c395f7cd42 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:12.302770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76ceb3-3daf-492e-a927-06c395f7cd42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:12.367852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76ceb3-3daf-492e-a927-06c395f7cd42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:12.369332 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76ceb3-3daf-492e-a927-06c395f7cd42 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:12.425526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:12.462127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 557/1128] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:39:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:39:12.468005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfaca6-ed6a-4489-9882-d672232d598a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:12.515154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:12.564517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfaca6-ed6a-4489-9882-d672232d598a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:12.596859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147b01a-4dcf-4325-8147-9bbf955623ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:12.597729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147b01a-4dcf-4325-8147-9bbf955623ed None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:12.598011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147b01a-4dcf-4325-8147-9bbf955623ed None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:12.598281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147b01a-4dcf-4325-8147-9bbf955623ed None admin] PATH_INFO: `/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:12.599466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147b01a-4dcf-4325-8147-9bbf955623ed None admin] RBAC: Authorizing `identity:get_project(project_id=09d82e96b9b04be4a0bf2190ef13a1dd)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:12.600902 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147b01a-4dcf-4325-8147-9bbf955623ed None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:12.602591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 572/1129] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:39:12 2022] GET /identity/v3/projects/09d82e96b9b04be4a0bf2190ef13a1dd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:12.608522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76ceb3-3daf-492e-a927-06c395f7cd42 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:12.702971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76ceb3-3daf-492e-a927-06c395f7cd42 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:12.735973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606c962-f352-45e1-a84e-00479fdf780b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:12.736612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606c962-f352-45e1-a84e-00479fdf780b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:12.736899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606c962-f352-45e1-a84e-00479fdf780b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:12.737178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606c962-f352-45e1-a84e-00479fdf780b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:12.738019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606c962-f352-45e1-a84e-00479fdf780b None admin] RBAC: Adding query filter params (role.id=5f24daf2ba2b4f8fab95c87cefb52231, scope.project.id=09d82e96b9b04be4a0bf2190ef13a1dd, user.id=312b2cc088c84ac3abfe650ee9e4e1a1) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:12.738333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606c962-f352-45e1-a84e-00479fdf780b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:12.739750 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606c962-f352-45e1-a84e-00479fdf780b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:12.745421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 558/1130] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:39:12 2022] GET /identity/v3/role_assignments?user.id=312b2cc088c84ac3abfe650ee9e4e1a1&scope.project.id=09d82e96b9b04be4a0bf2190ef13a1dd&role.id=5f24daf2ba2b4f8fab95c87cefb52231 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:14.843192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694604c-71ae-44e3-a04b-8cdabd7e1976 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:14.843730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694604c-71ae-44e3-a04b-8cdabd7e1976 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:14.843730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694604c-71ae-44e3-a04b-8cdabd7e1976 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.845684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 573/1131] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:14.850719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6df9c8-81b9-41fd-a4b6-60b66208931e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:14.851146 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6df9c8-81b9-41fd-a4b6-60b66208931e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:14.851285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6df9c8-81b9-41fd-a4b6-60b66208931e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:14.867922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-df6df9c8-81b9-41fd-a4b6-60b66208931e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:14.869104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-df6df9c8-81b9-41fd-a4b6-60b66208931e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:14.891272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:14.932499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 559/1132] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:14.955952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694604c-71ae-44e3-a04b-8cdabd7e1976 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:15.053799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694604c-71ae-44e3-a04b-8cdabd7e1976 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:15.069769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:15.086659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c2ecf-a120-45ec-9963-114c029b0c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:15.087435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c2ecf-a120-45ec-9963-114c029b0c49 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:15.087719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c2ecf-a120-45ec-9963-114c029b0c49 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:15.087991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c2ecf-a120-45ec-9963-114c029b0c49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:15.089186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c2ecf-a120-45ec-9963-114c029b0c49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:15.090656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c2ecf-a120-45ec-9963-114c029b0c49 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:15.092229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 574/1133] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:39:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:15.097985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6df9c8-81b9-41fd-a4b6-60b66208931e None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:15.193470 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6df9c8-81b9-41fd-a4b6-60b66208931e None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:15.225366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e4704-d4ed-495d-b134-eab7000c23f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:15.226007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e4704-d4ed-495d-b134-eab7000c23f4 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:15.226291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e4704-d4ed-495d-b134-eab7000c23f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:15.226565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e4704-d4ed-495d-b134-eab7000c23f4 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:15.227244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e4704-d4ed-495d-b134-eab7000c23f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:15.228665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e4704-d4ed-495d-b134-eab7000c23f4 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:15.313105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 560/1134] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:39:15 2022] POST /identity/v3/projects => generated 311 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:17.450918 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f44c8-6d06-493f-9def-93526faca342 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:17.451418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f44c8-6d06-493f-9def-93526faca342 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:17.451504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f44c8-6d06-493f-9def-93526faca342 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:17.452854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 575/1135] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:17.458047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851dd468-c374-4533-be28-7a94f5f51266 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:17.458423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851dd468-c374-4533-be28-7a94f5f51266 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:17.458615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851dd468-c374-4533-be28-7a94f5f51266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:17.476004 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-851dd468-c374-4533-be28-7a94f5f51266 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:17.477184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-851dd468-c374-4533-be28-7a94f5f51266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:17.479859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:17.541480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 561/1136] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:17.566781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f44c8-6d06-493f-9def-93526faca342 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:17.663504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f44c8-6d06-493f-9def-93526faca342 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:17.695953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5dc34-cefb-4dc4-9478-1adc75b4d29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:17.696726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5dc34-cefb-4dc4-9478-1adc75b4d29e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:17.697010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5dc34-cefb-4dc4-9478-1adc75b4d29e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:17.697311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5dc34-cefb-4dc4-9478-1adc75b4d29e None admin] PATH_INFO: `/v3/domains/default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:17.698474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5dc34-cefb-4dc4-9478-1adc75b4d29e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:17.699917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5dc34-cefb-4dc4-9478-1adc75b4d29e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:17.701524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 576/1137] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:39:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:17.707263 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851dd468-c374-4533-be28-7a94f5f51266 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:17.802068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851dd468-c374-4533-be28-7a94f5f51266 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:17.834302 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5c04a-634f-489b-9988-3e5c9c5054d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:17.834959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5c04a-634f-489b-9988-3e5c9c5054d1 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:17.835244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5c04a-634f-489b-9988-3e5c9c5054d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:17.835519 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5c04a-634f-489b-9988-3e5c9c5054d1 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:17.836224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5c04a-634f-489b-9988-3e5c9c5054d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:17.837687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5c04a-634f-489b-9988-3e5c9c5054d1 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:17.847413 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:17.924975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 562/1138] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:39:17 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:39:20.080343 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74667536-468b-4362-a369-b7b856da1408 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:20.080894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74667536-468b-4362-a369-b7b856da1408 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:20.080990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74667536-468b-4362-a369-b7b856da1408 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:20.082358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 577/1139] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:20.087582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63b266-292d-4f1d-a72f-7336d6c9fc7a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:20.087973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63b266-292d-4f1d-a72f-7336d6c9fc7a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:20.088148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63b266-292d-4f1d-a72f-7336d6c9fc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:20.104853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-4f63b266-292d-4f1d-a72f-7336d6c9fc7a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:20.107771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-4f63b266-292d-4f1d-a72f-7336d6c9fc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:20.171493 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 563/1140] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:20.195152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74667536-468b-4362-a369-b7b856da1408 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:20.247444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:20.293597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74667536-468b-4362-a369-b7b856da1408 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:20.325249 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921d07e-607a-450b-8b4d-f37a31dc2565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:20.326035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921d07e-607a-450b-8b4d-f37a31dc2565 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:20.326316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921d07e-607a-450b-8b4d-f37a31dc2565 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:20.326589 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921d07e-607a-450b-8b4d-f37a31dc2565 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:20.336286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921d07e-607a-450b-8b4d-f37a31dc2565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-8921d07e-607a-450b-8b4d-f37a31dc2565 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:39:20.343737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:39:20.344868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:39:20.345856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 578/1141] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:39:20.351338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63b266-292d-4f1d-a72f-7336d6c9fc7a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:20.444555 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63b266-292d-4f1d-a72f-7336d6c9fc7a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:20.468558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:20.477248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be2906-5586-4e29-bdea-cc1a2ef81fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:20.477929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be2906-5586-4e29-bdea-cc1a2ef81fee None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:20.480970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be2906-5586-4e29-bdea-cc1a2ef81fee None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:20.480970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be2906-5586-4e29-bdea-cc1a2ef81fee None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:20.480970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be2906-5586-4e29-bdea-cc1a2ef81fee None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:20.480970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be2906-5586-4e29-bdea-cc1a2ef81fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:20.480970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be2906-5586-4e29-bdea-cc1a2ef81fee None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:20.488457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 564/1142] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:39:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:20.494236 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921d07e-607a-450b-8b4d-f37a31dc2565 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:20.589555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921d07e-607a-450b-8b4d-f37a31dc2565 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:20.602053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:20.622190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999967b5-d163-48ff-abdb-fdfe332d5aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:20.623042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999967b5-d163-48ff-abdb-fdfe332d5aa4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:20.623324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999967b5-d163-48ff-abdb-fdfe332d5aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:20.623599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999967b5-d163-48ff-abdb-fdfe332d5aa4 None admin] PATH_INFO: `/v3/users/a37fadb0ad59451cb8e72001398d6713` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:20.636960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999967b5-d163-48ff-abdb-fdfe332d5aa4 None admin] RBAC: Authorizing `identity:get_user(user_id=a37fadb0ad59451cb8e72001398d6713)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:20.638423 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999967b5-d163-48ff-abdb-fdfe332d5aa4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:20.640037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 579/1143] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:20 2022] GET /identity/v3/users/a37fadb0ad59451cb8e72001398d6713 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:20.646167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be2906-5586-4e29-bdea-cc1a2ef81fee None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:20.741626 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be2906-5586-4e29-bdea-cc1a2ef81fee None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:20.773932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac386b-9017-4d5c-b0ac-fc9655ac7654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:20.774581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac386b-9017-4d5c-b0ac-fc9655ac7654 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:20.774844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac386b-9017-4d5c-b0ac-fc9655ac7654 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:20.775117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac386b-9017-4d5c-b0ac-fc9655ac7654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:20.828569 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:20.840188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac386b-9017-4d5c-b0ac-fc9655ac7654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:20.841578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac386b-9017-4d5c-b0ac-fc9655ac7654 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:20.935802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 565/1144] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:39:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:39:20.941651 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999967b5-d163-48ff-abdb-fdfe332d5aa4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:21.034869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999967b5-d163-48ff-abdb-fdfe332d5aa4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:21.068497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364a9a2-89b5-4821-8cc1-b539afbcaecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:21.069358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364a9a2-89b5-4821-8cc1-b539afbcaecf None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:21.069640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364a9a2-89b5-4821-8cc1-b539afbcaecf None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:21.069908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364a9a2-89b5-4821-8cc1-b539afbcaecf None admin] PATH_INFO: `/v3/projects/421af71b855c446ea4da39f8449ba390` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:21.071052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364a9a2-89b5-4821-8cc1-b539afbcaecf None admin] RBAC: Authorizing `identity:get_project(project_id=421af71b855c446ea4da39f8449ba390)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:21.072459 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364a9a2-89b5-4821-8cc1-b539afbcaecf None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:21.074124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 580/1145] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:39:20 2022] GET /identity/v3/projects/421af71b855c446ea4da39f8449ba390 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:21.080035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac386b-9017-4d5c-b0ac-fc9655ac7654 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:21.161837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:21.173688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac386b-9017-4d5c-b0ac-fc9655ac7654 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:21.206095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca127d6-e9b0-49b1-9660-c8d4c5ed163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:21.206733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca127d6-e9b0-49b1-9660-c8d4c5ed163b None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:21.207020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca127d6-e9b0-49b1-9660-c8d4c5ed163b None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:21.207299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca127d6-e9b0-49b1-9660-c8d4c5ed163b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:21.208109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca127d6-e9b0-49b1-9660-c8d4c5ed163b None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=421af71b855c446ea4da39f8449ba390, user.id=a37fadb0ad59451cb8e72001398d6713) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:21.208422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca127d6-e9b0-49b1-9660-c8d4c5ed163b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:21.210401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca127d6-e9b0-49b1-9660-c8d4c5ed163b None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:21.215623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 566/1146] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:39:21 2022] GET /identity/v3/role_assignments?user.id=a37fadb0ad59451cb8e72001398d6713&scope.project.id=421af71b855c446ea4da39f8449ba390&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:23.289147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38b117-e42a-407d-9c48-5b5c8bf8f30b None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:23.289701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38b117-e42a-407d-9c48-5b5c8bf8f30b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:23.289813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38b117-e42a-407d-9c48-5b5c8bf8f30b None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:23.291100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 581/1147] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:23.296309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b50815-b86c-4b47-a0f3-bc33ff887b5a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:23.296739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b50815-b86c-4b47-a0f3-bc33ff887b5a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:23.296923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b50815-b86c-4b47-a0f3-bc33ff887b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:23.313744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-68b50815-b86c-4b47-a0f3-bc33ff887b5a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:23.314933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-68b50815-b86c-4b47-a0f3-bc33ff887b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:23.379231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 567/1148] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:23.401535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38b117-e42a-407d-9c48-5b5c8bf8f30b None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:23.441859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:23.499226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38b117-e42a-407d-9c48-5b5c8bf8f30b None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:23.531452 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a88100-5eb5-4796-91ad-2d959306ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:23.532226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a88100-5eb5-4796-91ad-2d959306ce3a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:23.532513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a88100-5eb5-4796-91ad-2d959306ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:23.532791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a88100-5eb5-4796-91ad-2d959306ce3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:23.543483 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a88100-5eb5-4796-91ad-2d959306ce3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-03a88100-5eb5-4796-91ad-2d959306ce3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:39:23.550979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:39:23.552094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:39:23.553073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 582/1149] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:39:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:39:23.558201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b50815-b86c-4b47-a0f3-bc33ff887b5a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:23.636695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:23.654366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b50815-b86c-4b47-a0f3-bc33ff887b5a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:23.687608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160a89a-e96a-492c-9ac6-accc7510bd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:23.688272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160a89a-e96a-492c-9ac6-accc7510bd4c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:23.688557 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160a89a-e96a-492c-9ac6-accc7510bd4c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:23.688838 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160a89a-e96a-492c-9ac6-accc7510bd4c None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:23.689614 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160a89a-e96a-492c-9ac6-accc7510bd4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:23.689923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160a89a-e96a-492c-9ac6-accc7510bd4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:23.691296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160a89a-e96a-492c-9ac6-accc7510bd4c None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:23.699055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 568/1150] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:39:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:23.705332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a88100-5eb5-4796-91ad-2d959306ce3a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:23.797772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:23.801359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a88100-5eb5-4796-91ad-2d959306ce3a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:23.834206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4fe7b-5d57-4112-ace7-aeddf53ece77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:23.835054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4fe7b-5d57-4112-ace7-aeddf53ece77 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:23.835336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4fe7b-5d57-4112-ace7-aeddf53ece77 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:23.835609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4fe7b-5d57-4112-ace7-aeddf53ece77 None admin] PATH_INFO: `/v3/users/a37fadb0ad59451cb8e72001398d6713` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:23.836764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4fe7b-5d57-4112-ace7-aeddf53ece77 None admin] RBAC: Authorizing `identity:get_user(user_id=a37fadb0ad59451cb8e72001398d6713)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:23.838218 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4fe7b-5d57-4112-ace7-aeddf53ece77 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:23.839814 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 583/1151] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:23 2022] GET /identity/v3/users/a37fadb0ad59451cb8e72001398d6713 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:23.845489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160a89a-e96a-492c-9ac6-accc7510bd4c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:23.940991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160a89a-e96a-492c-9ac6-accc7510bd4c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:23.973144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ead58-ae17-4636-b325-942c3e8cd303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:23.974079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ead58-ae17-4636-b325-942c3e8cd303 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:23.974362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ead58-ae17-4636-b325-942c3e8cd303 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:23.974638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ead58-ae17-4636-b325-942c3e8cd303 None admin] PATH_INFO: `/v3/projects/421af71b855c446ea4da39f8449ba390` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:23.975823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ead58-ae17-4636-b325-942c3e8cd303 None admin] RBAC: Authorizing `identity:get_project(project_id=421af71b855c446ea4da39f8449ba390)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:23.977307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ead58-ae17-4636-b325-942c3e8cd303 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:23.979003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 569/1152] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:39:23 2022] GET /identity/v3/projects/421af71b855c446ea4da39f8449ba390 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:23.984983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4fe7b-5d57-4112-ace7-aeddf53ece77 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:24.082613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4fe7b-5d57-4112-ace7-aeddf53ece77 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:24.114813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e8df5-006c-45ae-a4ad-db87ef12b7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:24.115686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e8df5-006c-45ae-a4ad-db87ef12b7bf None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:24.115968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e8df5-006c-45ae-a4ad-db87ef12b7bf None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:24.116241 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e8df5-006c-45ae-a4ad-db87ef12b7bf None admin] PATH_INFO: `/v3/projects/421af71b855c446ea4da39f8449ba390/users/a37fadb0ad59451cb8e72001398d6713/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:24.118191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e8df5-006c-45ae-a4ad-db87ef12b7bf None admin] RBAC: Authorizing `identity:create_grant(project_id=421af71b855c446ea4da39f8449ba390, user_id=a37fadb0ad59451cb8e72001398d6713, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:24.119654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e8df5-006c-45ae-a4ad-db87ef12b7bf None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:24.211979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 584/1153] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:39:23 2022] PUT /identity/v3/projects/421af71b855c446ea4da39f8449ba390/users/a37fadb0ad59451cb8e72001398d6713/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:26.295703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62038d99-c48b-48f3-a64c-15dac393165f None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:26.339407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62038d99-c48b-48f3-a64c-15dac393165f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:26.339407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62038d99-c48b-48f3-a64c-15dac393165f None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:26.339407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 570/1154] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:26.339714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d780b4-d284-42e2-8afb-82f2d708c8e5 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:26.339714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d780b4-d284-42e2-8afb-82f2d708c8e5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:26.339714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d780b4-d284-42e2-8afb-82f2d708c8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:26.339714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-a8d780b4-d284-42e2-8afb-82f2d708c8e5 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:26.339714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a8d780b4-d284-42e2-8afb-82f2d708c8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:26.340379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:26.385458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 585/1155] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:26.409066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62038d99-c48b-48f3-a64c-15dac393165f None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:26.424413 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:26.506893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62038d99-c48b-48f3-a64c-15dac393165f None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:26.539869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb5e6c-1466-4e51-abe6-f515c006978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:26.540642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb5e6c-1466-4e51-abe6-f515c006978d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:26.540927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb5e6c-1466-4e51-abe6-f515c006978d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:26.541203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb5e6c-1466-4e51-abe6-f515c006978d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:26.550973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb5e6c-1466-4e51-abe6-f515c006978d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-08cb5e6c-1466-4e51-abe6-f515c006978d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:39:26.558472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:39:26.559601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:39:26.560687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 571/1156] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:39:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:39:26.565576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d780b4-d284-42e2-8afb-82f2d708c8e5 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:26.659074 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d780b4-d284-42e2-8afb-82f2d708c8e5 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:26.691164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027730d-15e1-413d-920e-1dc8cc1be840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:26.694761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027730d-15e1-413d-920e-1dc8cc1be840 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:26.694761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027730d-15e1-413d-920e-1dc8cc1be840 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:26.694761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027730d-15e1-413d-920e-1dc8cc1be840 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:26.694761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027730d-15e1-413d-920e-1dc8cc1be840 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:26.694761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027730d-15e1-413d-920e-1dc8cc1be840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:26.694761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027730d-15e1-413d-920e-1dc8cc1be840 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:26.703246 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 586/1157] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:39:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:26.709000 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb5e6c-1466-4e51-abe6-f515c006978d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:26.775907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:26.804939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb5e6c-1466-4e51-abe6-f515c006978d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:26.837356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ca2a3-4a2f-499a-8cb4-e6aa91ca0789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:26.838210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ca2a3-4a2f-499a-8cb4-e6aa91ca0789 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:26.838491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ca2a3-4a2f-499a-8cb4-e6aa91ca0789 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:26.838762 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ca2a3-4a2f-499a-8cb4-e6aa91ca0789 None admin] PATH_INFO: `/v3/users/a37fadb0ad59451cb8e72001398d6713` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:26.839921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ca2a3-4a2f-499a-8cb4-e6aa91ca0789 None admin] RBAC: Authorizing `identity:get_user(user_id=a37fadb0ad59451cb8e72001398d6713)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:26.841382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ca2a3-4a2f-499a-8cb4-e6aa91ca0789 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:26.842983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 572/1158] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:26 2022] GET /identity/v3/users/a37fadb0ad59451cb8e72001398d6713 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:26.848553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027730d-15e1-413d-920e-1dc8cc1be840 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:26.867849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:26.945938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027730d-15e1-413d-920e-1dc8cc1be840 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:26.979173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e7161-ab2a-4e9c-a721-c8681f291395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:26.979823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e7161-ab2a-4e9c-a721-c8681f291395 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:26.980107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e7161-ab2a-4e9c-a721-c8681f291395 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:26.980414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e7161-ab2a-4e9c-a721-c8681f291395 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:27.046408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e7161-ab2a-4e9c-a721-c8681f291395 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:27.047870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e7161-ab2a-4e9c-a721-c8681f291395 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:27.065030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:27.142927 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 587/1159] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:39:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:39:27.148799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ca2a3-4a2f-499a-8cb4-e6aa91ca0789 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:27.244185 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ca2a3-4a2f-499a-8cb4-e6aa91ca0789 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:27.275964 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23ded5-211a-4157-93b1-58ae3d29f981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:27.276804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23ded5-211a-4157-93b1-58ae3d29f981 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:27.277089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23ded5-211a-4157-93b1-58ae3d29f981 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:27.277417 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23ded5-211a-4157-93b1-58ae3d29f981 None admin] PATH_INFO: `/v3/projects/421af71b855c446ea4da39f8449ba390` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:27.278571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23ded5-211a-4157-93b1-58ae3d29f981 None admin] RBAC: Authorizing `identity:get_project(project_id=421af71b855c446ea4da39f8449ba390)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:27.280262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23ded5-211a-4157-93b1-58ae3d29f981 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:27.281946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 573/1160] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:39:27 2022] GET /identity/v3/projects/421af71b855c446ea4da39f8449ba390 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:27.287795 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e7161-ab2a-4e9c-a721-c8681f291395 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:27.381794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e7161-ab2a-4e9c-a721-c8681f291395 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:27.397540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:27.413385 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4e977-95be-418c-8135-6d08da8b9a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:27.416242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4e977-95be-418c-8135-6d08da8b9a53 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:27.416242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4e977-95be-418c-8135-6d08da8b9a53 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:27.416242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4e977-95be-418c-8135-6d08da8b9a53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:27.416242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4e977-95be-418c-8135-6d08da8b9a53 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=421af71b855c446ea4da39f8449ba390, user.id=a37fadb0ad59451cb8e72001398d6713) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:27.416242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4e977-95be-418c-8135-6d08da8b9a53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:27.417027 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4e977-95be-418c-8135-6d08da8b9a53 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:27.423070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 588/1161] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:39:27 2022] GET /identity/v3/role_assignments?user.id=a37fadb0ad59451cb8e72001398d6713&scope.project.id=421af71b855c446ea4da39f8449ba390&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:29.530552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575adb8-6627-4f0f-a441-10e5786a5794 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:29.531089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575adb8-6627-4f0f-a441-10e5786a5794 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:29.531089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575adb8-6627-4f0f-a441-10e5786a5794 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:29.532462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 574/1162] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:29.537557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85244926-2d65-420f-a2dd-f3aee05d4b29 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:29.537953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85244926-2d65-420f-a2dd-f3aee05d4b29 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:29.538110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85244926-2d65-420f-a2dd-f3aee05d4b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:29.554168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-85244926-2d65-420f-a2dd-f3aee05d4b29 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:29.555359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-85244926-2d65-420f-a2dd-f3aee05d4b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:29.619438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 589/1163] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:39:29.641660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575adb8-6627-4f0f-a441-10e5786a5794 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:29.653512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:29.737930 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575adb8-6627-4f0f-a441-10e5786a5794 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:29.769453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8ca5a-696b-41aa-8dee-2ff138f1c880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:29.770083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8ca5a-696b-41aa-8dee-2ff138f1c880 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:29.770361 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8ca5a-696b-41aa-8dee-2ff138f1c880 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:29.770629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8ca5a-696b-41aa-8dee-2ff138f1c880 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:29.833548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8ca5a-696b-41aa-8dee-2ff138f1c880 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:29.834973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8ca5a-696b-41aa-8dee-2ff138f1c880 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:29.843814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:29.927359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 575/1164] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:39:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:39:29.933132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85244926-2d65-420f-a2dd-f3aee05d4b29 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:30.027502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85244926-2d65-420f-a2dd-f3aee05d4b29 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:30.040397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:30.060050 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252bbac-ae94-4014-9dcd-78b8e8466e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:30.060833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252bbac-ae94-4014-9dcd-78b8e8466e03 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:30.061114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252bbac-ae94-4014-9dcd-78b8e8466e03 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:30.061416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252bbac-ae94-4014-9dcd-78b8e8466e03 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:30.068002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252bbac-ae94-4014-9dcd-78b8e8466e03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:30.069759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a252bbac-ae94-4014-9dcd-78b8e8466e03 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-a252bbac-ae94-4014-9dcd-78b8e8466e03 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:39:30.083510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:39:30.084721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:39:30.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 590/1165] 10.222.0.81 () {56 vars in 1117 bytes} [Thu Jun 16 18:39:29 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:39:30.091103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8ca5a-696b-41aa-8dee-2ff138f1c880 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:30.195715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8ca5a-696b-41aa-8dee-2ff138f1c880 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:30.204588 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:30.228886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb5af1-7911-430c-b91f-f5b5f04fb0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:30.229571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb5af1-7911-430c-b91f-f5b5f04fb0ab None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:30.229859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb5af1-7911-430c-b91f-f5b5f04fb0ab None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:30.230137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb5af1-7911-430c-b91f-f5b5f04fb0ab None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:30.230839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb5af1-7911-430c-b91f-f5b5f04fb0ab None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:30.231148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb5af1-7911-430c-b91f-f5b5f04fb0ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:30.232546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb5af1-7911-430c-b91f-f5b5f04fb0ab None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:30.239407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 576/1166] 10.222.0.81 () {56 vars in 1115 bytes} [Thu Jun 16 18:39:30 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:32.306559 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9330a-9180-44cb-afc3-841b6f60900e None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:32.307086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9330a-9180-44cb-afc3-841b6f60900e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:32.307177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9330a-9180-44cb-afc3-841b6f60900e None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:32.308471 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 591/1167] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:32.313519 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b917-df96-425e-9d12-b13730ba4d14 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:32.313891 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b917-df96-425e-9d12-b13730ba4d14 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:32.314082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2b917-df96-425e-9d12-b13730ba4d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:32.330198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d2c2b917-df96-425e-9d12-b13730ba4d14 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:32.331362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d2c2b917-df96-425e-9d12-b13730ba4d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:32.394520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 577/1168] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:32.418368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9330a-9180-44cb-afc3-841b6f60900e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:32.515515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9330a-9180-44cb-afc3-841b6f60900e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:32.547450 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f307e-c161-4b5e-a8bc-0bed24f5127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:32.548091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f307e-c161-4b5e-a8bc-0bed24f5127f None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:32.548370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f307e-c161-4b5e-a8bc-0bed24f5127f None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:32.548640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f307e-c161-4b5e-a8bc-0bed24f5127f None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:32.549393 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f307e-c161-4b5e-a8bc-0bed24f5127f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:32.550746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f307e-c161-4b5e-a8bc-0bed24f5127f None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:32.628978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 592/1169] 10.222.0.81 () {60 vars in 1138 bytes} [Thu Jun 16 18:39:32 2022] POST /identity/v3/domains => generated 267 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:34.749166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662178-89e2-4108-8d16-25af21a9516a None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:34.749743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662178-89e2-4108-8d16-25af21a9516a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:34.749743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc662178-89e2-4108-8d16-25af21a9516a None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:34.751029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 578/1170] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:34.756785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796598e-89ed-4cd2-a67a-b7c6ef66db3e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:34.757207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796598e-89ed-4cd2-a67a-b7c6ef66db3e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:34.757342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d796598e-89ed-4cd2-a67a-b7c6ef66db3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:34.774183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d796598e-89ed-4cd2-a67a-b7c6ef66db3e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:34.775341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d796598e-89ed-4cd2-a67a-b7c6ef66db3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:34.799588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:34.838214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 593/1171] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:34.861433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc662178-89e2-4108-8d16-25af21a9516a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:34.956791 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc662178-89e2-4108-8d16-25af21a9516a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:34.958270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:34.988504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c1fbb-cf02-42c0-8824-585271b1ca31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:34.989370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c1fbb-cf02-42c0-8824-585271b1ca31 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:34.989652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c1fbb-cf02-42c0-8824-585271b1ca31 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:34.989924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c1fbb-cf02-42c0-8824-585271b1ca31 None admin] PATH_INFO: `/v3/domains/c70f7ca7af484d06a8dc5a5023ec00c1` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:34.999712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c1fbb-cf02-42c0-8824-585271b1ca31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c70f7ca7af484d06a8dc5a5023ec00c1)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:35.001650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c1fbb-cf02-42c0-8824-585271b1ca31 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:35.003229 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 579/1172] 10.222.0.81 () {56 vars in 1183 bytes} [Thu Jun 16 18:39:34 2022] GET /identity/v3/domains/c70f7ca7af484d06a8dc5a5023ec00c1 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:35.008944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796598e-89ed-4cd2-a67a-b7c6ef66db3e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:35.102742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d796598e-89ed-4cd2-a67a-b7c6ef66db3e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:35.135108 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355336a6-d630-4f96-8d02-830bedb81861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:35.135743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355336a6-d630-4f96-8d02-830bedb81861 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:35.136022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355336a6-d630-4f96-8d02-830bedb81861 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:35.136293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355336a6-d630-4f96-8d02-830bedb81861 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:35.136974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355336a6-d630-4f96-8d02-830bedb81861 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:35.138419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355336a6-d630-4f96-8d02-830bedb81861 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:35.212069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 594/1173] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:39:35 2022] POST /identity/v3/projects => generated 361 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:37.329597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8752af-b805-46e3-ac53-c0b76734d90e None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:37.330147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8752af-b805-46e3-ac53-c0b76734d90e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:37.330147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8752af-b805-46e3-ac53-c0b76734d90e None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:37.331524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 580/1174] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:37.336532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a064-7144-43be-b745-11729d99ee9a None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:37.336949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a064-7144-43be-b745-11729d99ee9a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:37.337084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a064-7144-43be-b745-11729d99ee9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:37.353826 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-83e6a064-7144-43be-b745-11729d99ee9a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:37.354992 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-83e6a064-7144-43be-b745-11729d99ee9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:37.363940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:37.419391 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 595/1175] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:37.444682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8752af-b805-46e3-ac53-c0b76734d90e None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:37.542508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8752af-b805-46e3-ac53-c0b76734d90e None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:37.575422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e7aae-adab-45ae-846f-fb72a33c2bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:37.576301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e7aae-adab-45ae-846f-fb72a33c2bf7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:37.576585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e7aae-adab-45ae-846f-fb72a33c2bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:37.576863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e7aae-adab-45ae-846f-fb72a33c2bf7 None admin] PATH_INFO: `/v3/domains/c70f7ca7af484d06a8dc5a5023ec00c1` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:37.578051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e7aae-adab-45ae-846f-fb72a33c2bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c70f7ca7af484d06a8dc5a5023ec00c1)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:37.579485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92e7aae-adab-45ae-846f-fb72a33c2bf7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:37.581128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 581/1176] 10.222.0.81 () {56 vars in 1183 bytes} [Thu Jun 16 18:39:37 2022] GET /identity/v3/domains/c70f7ca7af484d06a8dc5a5023ec00c1 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:37.586568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a064-7144-43be-b745-11729d99ee9a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:37.681199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a064-7144-43be-b745-11729d99ee9a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:37.713394 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05bf48-9d7b-4b37-9271-8798b75484a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:37.714040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05bf48-9d7b-4b37-9271-8798b75484a1 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:37.714325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05bf48-9d7b-4b37-9271-8798b75484a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:37.714603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05bf48-9d7b-4b37-9271-8798b75484a1 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:37.715289 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05bf48-9d7b-4b37-9271-8798b75484a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:37.717263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05bf48-9d7b-4b37-9271-8798b75484a1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:37.816664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:37.821277 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 596/1177] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:39:37 2022] POST /identity/v3/users => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:39:39.977556 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd11c8f-2f98-4069-931f-2a4b67789764 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:39.978201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd11c8f-2f98-4069-931f-2a4b67789764 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:39.978201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd11c8f-2f98-4069-931f-2a4b67789764 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:39.979990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 582/1178] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:39.985201 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e12cd-6e00-4f99-ae24-ab2b251dcebb None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:39.985608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e12cd-6e00-4f99-ae24-ab2b251dcebb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:39.985830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e12cd-6e00-4f99-ae24-ab2b251dcebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:40.002061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-2a5e12cd-6e00-4f99-ae24-ab2b251dcebb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:40.003236 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-2a5e12cd-6e00-4f99-ae24-ab2b251dcebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:40.067752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 597/1179] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:40.091487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd11c8f-2f98-4069-931f-2a4b67789764 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:40.129871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:40.204067 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd11c8f-2f98-4069-931f-2a4b67789764 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:40.236655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e9581-57d6-4690-9697-54009018bcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:40.237933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e9581-57d6-4690-9697-54009018bcfb None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:40.238450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e9581-57d6-4690-9697-54009018bcfb None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:40.238941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e9581-57d6-4690-9697-54009018bcfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:40.254652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e9581-57d6-4690-9697-54009018bcfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-2b0e9581-57d6-4690-9697-54009018bcfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:39:40.263007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:39:40.264164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:39:40.265108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 583/1180] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:39:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:39:40.270194 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e12cd-6e00-4f99-ae24-ab2b251dcebb None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:40.365207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e12cd-6e00-4f99-ae24-ab2b251dcebb None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:40.393794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:40.399319 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5e52b-6276-42e5-a6d7-092e15923789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:40.399999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e52b-6276-42e5-a6d7-092e15923789 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:40.400280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e52b-6276-42e5-a6d7-092e15923789 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:40.400555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5e52b-6276-42e5-a6d7-092e15923789 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:40.401346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5e52b-6276-42e5-a6d7-092e15923789 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:40.407467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5e52b-6276-42e5-a6d7-092e15923789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:40.407467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5e52b-6276-42e5-a6d7-092e15923789 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:40.410807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 598/1181] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:39:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:40.417142 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e9581-57d6-4690-9697-54009018bcfb None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:40.513319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:40.522874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e9581-57d6-4690-9697-54009018bcfb None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:40.555395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d98fc-5c25-493b-a672-0e23253598bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:40.556803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d98fc-5c25-493b-a672-0e23253598bc None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:40.557084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d98fc-5c25-493b-a672-0e23253598bc None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:40.557391 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d98fc-5c25-493b-a672-0e23253598bc None admin] PATH_INFO: `/v3/users/3d3b100de68443f28966bb11a85ca0d3` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:40.569845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d98fc-5c25-493b-a672-0e23253598bc None admin] RBAC: Authorizing `identity:get_user(user_id=3d3b100de68443f28966bb11a85ca0d3)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:40.571282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d98fc-5c25-493b-a672-0e23253598bc None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:40.572992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 584/1182] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:40 2022] GET /identity/v3/users/3d3b100de68443f28966bb11a85ca0d3 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:40.578821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5e52b-6276-42e5-a6d7-092e15923789 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:40.674216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5e52b-6276-42e5-a6d7-092e15923789 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:40.706529 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e39187-7a49-452a-a86b-8e2e4f273f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:40.707191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e39187-7a49-452a-a86b-8e2e4f273f59 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:40.707475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e39187-7a49-452a-a86b-8e2e4f273f59 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:40.707749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e39187-7a49-452a-a86b-8e2e4f273f59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:40.762730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:40.771638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e39187-7a49-452a-a86b-8e2e4f273f59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:40.773023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e39187-7a49-452a-a86b-8e2e4f273f59 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:40.877960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 599/1183] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:39:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:39:40.884619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d98fc-5c25-493b-a672-0e23253598bc None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:40.977684 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d98fc-5c25-493b-a672-0e23253598bc None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:41.010158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb991595-ca53-4d99-919c-63265879e811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:41.011252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb991595-ca53-4d99-919c-63265879e811 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:41.011532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb991595-ca53-4d99-919c-63265879e811 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:41.011804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb991595-ca53-4d99-919c-63265879e811 None admin] PATH_INFO: `/v3/projects/b64ee1ed315e424e8963398f9560b109` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:41.012943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb991595-ca53-4d99-919c-63265879e811 None admin] RBAC: Authorizing `identity:get_project(project_id=b64ee1ed315e424e8963398f9560b109)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:41.014400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb991595-ca53-4d99-919c-63265879e811 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:41.016040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 585/1184] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:39:40 2022] GET /identity/v3/projects/b64ee1ed315e424e8963398f9560b109 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:41.022079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e39187-7a49-452a-a86b-8e2e4f273f59 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:41.108507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:41.116565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e39187-7a49-452a-a86b-8e2e4f273f59 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:41.150243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb48975-f14f-4014-8581-d85dd3ba9ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:41.150898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb48975-f14f-4014-8581-d85dd3ba9ad5 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:41.151185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb48975-f14f-4014-8581-d85dd3ba9ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:41.151462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb48975-f14f-4014-8581-d85dd3ba9ad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:41.152284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb48975-f14f-4014-8581-d85dd3ba9ad5 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=b64ee1ed315e424e8963398f9560b109, user.id=3d3b100de68443f28966bb11a85ca0d3) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:41.152598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb48975-f14f-4014-8581-d85dd3ba9ad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:41.154065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb48975-f14f-4014-8581-d85dd3ba9ad5 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:41.159947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 600/1185] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:39:41 2022] GET /identity/v3/role_assignments?user.id=3d3b100de68443f28966bb11a85ca0d3&scope.project.id=b64ee1ed315e424e8963398f9560b109&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:43.227733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bfa2a-f8a3-4886-bb35-748327d70e00 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:43.228264 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bfa2a-f8a3-4886-bb35-748327d70e00 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:43.228264 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82bfa2a-f8a3-4886-bb35-748327d70e00 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:43.229667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 586/1186] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:43.234802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a9adc-8b59-4184-8e8e-796cbde959ec None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:43.235197 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a9adc-8b59-4184-8e8e-796cbde959ec None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:43.235359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a9adc-8b59-4184-8e8e-796cbde959ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:43.251607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-988a9adc-8b59-4184-8e8e-796cbde959ec None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:43.252786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-988a9adc-8b59-4184-8e8e-796cbde959ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:43.316544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 601/1187] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:39:43.339231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bfa2a-f8a3-4886-bb35-748327d70e00 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:43.356562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:43.436429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82bfa2a-f8a3-4886-bb35-748327d70e00 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:43.469351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc71d97-169b-4b33-b11a-41084b9ff83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:43.470126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc71d97-169b-4b33-b11a-41084b9ff83c None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:43.470411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc71d97-169b-4b33-b11a-41084b9ff83c None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:43.470685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc71d97-169b-4b33-b11a-41084b9ff83c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:43.480474 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc71d97-169b-4b33-b11a-41084b9ff83c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-dfc71d97-169b-4b33-b11a-41084b9ff83c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:39:43.487885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:39:43.488996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:39:43.490248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 587/1188] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:39:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:39:43.495241 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a9adc-8b59-4184-8e8e-796cbde959ec None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:43.576206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:43.590718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a9adc-8b59-4184-8e8e-796cbde959ec None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:43.622439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a1322-4849-4ab8-9279-0982f4a00def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:43.623082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a1322-4849-4ab8-9279-0982f4a00def None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:43.623366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a1322-4849-4ab8-9279-0982f4a00def None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:43.623637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a1322-4849-4ab8-9279-0982f4a00def None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:43.624356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a1322-4849-4ab8-9279-0982f4a00def None admin] RBAC: Adding query filter params (name=admin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:43.624663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a1322-4849-4ab8-9279-0982f4a00def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:43.626048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a1322-4849-4ab8-9279-0982f4a00def None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:43.634515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 602/1189] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:39:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:43.640109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc71d97-169b-4b33-b11a-41084b9ff83c None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:43.714742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:43.736035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc71d97-169b-4b33-b11a-41084b9ff83c None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:43.767245 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bcdc-6954-4fb3-bb3b-e73cc86f2155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:43.768094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bcdc-6954-4fb3-bb3b-e73cc86f2155 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:43.768372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bcdc-6954-4fb3-bb3b-e73cc86f2155 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:43.768642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97bcdc-6954-4fb3-bb3b-e73cc86f2155 None admin] PATH_INFO: `/v3/users/3d3b100de68443f28966bb11a85ca0d3` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:43.769813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97bcdc-6954-4fb3-bb3b-e73cc86f2155 None admin] RBAC: Authorizing `identity:get_user(user_id=3d3b100de68443f28966bb11a85ca0d3)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:43.771784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97bcdc-6954-4fb3-bb3b-e73cc86f2155 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:43.773375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 588/1190] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:43 2022] GET /identity/v3/users/3d3b100de68443f28966bb11a85ca0d3 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:43.779012 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a1322-4849-4ab8-9279-0982f4a00def None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:43.875179 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a1322-4849-4ab8-9279-0982f4a00def None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:43.908271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892b7ff-2be8-4255-8918-d5ad830c94a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:43.909126 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892b7ff-2be8-4255-8918-d5ad830c94a6 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:43.909439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892b7ff-2be8-4255-8918-d5ad830c94a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:43.909714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892b7ff-2be8-4255-8918-d5ad830c94a6 None admin] PATH_INFO: `/v3/projects/b64ee1ed315e424e8963398f9560b109` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:43.910875 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892b7ff-2be8-4255-8918-d5ad830c94a6 None admin] RBAC: Authorizing `identity:get_project(project_id=b64ee1ed315e424e8963398f9560b109)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:43.912904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892b7ff-2be8-4255-8918-d5ad830c94a6 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:43.914621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 603/1191] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:39:43 2022] GET /identity/v3/projects/b64ee1ed315e424e8963398f9560b109 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:43.920499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bcdc-6954-4fb3-bb3b-e73cc86f2155 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:44.016513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97bcdc-6954-4fb3-bb3b-e73cc86f2155 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:44.048681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34575c-dacc-4213-8a51-0157795db9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:44.049575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34575c-dacc-4213-8a51-0157795db9b7 None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:44.049858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34575c-dacc-4213-8a51-0157795db9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:44.050135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34575c-dacc-4213-8a51-0157795db9b7 None admin] PATH_INFO: `/v3/projects/b64ee1ed315e424e8963398f9560b109/users/3d3b100de68443f28966bb11a85ca0d3/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:44.052061 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34575c-dacc-4213-8a51-0157795db9b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b64ee1ed315e424e8963398f9560b109, user_id=3d3b100de68443f28966bb11a85ca0d3, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:44.053503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34575c-dacc-4213-8a51-0157795db9b7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:44.115838 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 589/1192] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:39:43 2022] PUT /identity/v3/projects/b64ee1ed315e424e8963398f9560b109/users/3d3b100de68443f28966bb11a85ca0d3/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:39:46.211081 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b54a0-c5b1-45b4-a5cd-87726f1233fc None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.211637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b54a0-c5b1-45b4-a5cd-87726f1233fc None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.211637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b54a0-c5b1-45b4-a5cd-87726f1233fc None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.212976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 604/1193] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:39:46.217980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf180d-24c9-4499-b42c-21de70fbdab4 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.218363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf180d-24c9-4499-b42c-21de70fbdab4 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.218528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf180d-24c9-4499-b42c-21de70fbdab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.224927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:46.235480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2bdf180d-24c9-4499-b42c-21de70fbdab4 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:39:46.240150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2bdf180d-24c9-4499-b42c-21de70fbdab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:39:46.308561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 590/1194] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:39:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:39:46.334378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b54a0-c5b1-45b4-a5cd-87726f1233fc None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.352384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:46.433962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b54a0-c5b1-45b4-a5cd-87726f1233fc None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.466961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bbe39-a5ac-46d0-b4e6-2d0f0836759e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.467677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bbe39-a5ac-46d0-b4e6-2d0f0836759e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.467961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bbe39-a5ac-46d0-b4e6-2d0f0836759e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.468234 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bbe39-a5ac-46d0-b4e6-2d0f0836759e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.477999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bbe39-a5ac-46d0-b4e6-2d0f0836759e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-e93bbe39-a5ac-46d0-b4e6-2d0f0836759e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:39:46.485538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:39:46.486648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:39:46.487682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 605/1195] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:39:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:39:46.492822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf180d-24c9-4499-b42c-21de70fbdab4 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.588532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf180d-24c9-4499-b42c-21de70fbdab4 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.620151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551ce14-fc7a-4e99-9f8e-54da3131ce46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.620803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551ce14-fc7a-4e99-9f8e-54da3131ce46 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.621087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551ce14-fc7a-4e99-9f8e-54da3131ce46 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.621394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551ce14-fc7a-4e99-9f8e-54da3131ce46 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.622122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551ce14-fc7a-4e99-9f8e-54da3131ce46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:46.622431 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551ce14-fc7a-4e99-9f8e-54da3131ce46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.623803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c551ce14-fc7a-4e99-9f8e-54da3131ce46 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.625117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:46.632513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 591/1196] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:46.638374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bbe39-a5ac-46d0-b4e6-2d0f0836759e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.713843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:46.735932 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bbe39-a5ac-46d0-b4e6-2d0f0836759e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.768742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7eca9-c0b8-4279-a800-ee0c88d66deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.769612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7eca9-c0b8-4279-a800-ee0c88d66deb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.769892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7eca9-c0b8-4279-a800-ee0c88d66deb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.770161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7eca9-c0b8-4279-a800-ee0c88d66deb None admin] PATH_INFO: `/v3/users/3d3b100de68443f28966bb11a85ca0d3` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.771312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7eca9-c0b8-4279-a800-ee0c88d66deb None admin] RBAC: Authorizing `identity:get_user(user_id=3d3b100de68443f28966bb11a85ca0d3)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.772723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7eca9-c0b8-4279-a800-ee0c88d66deb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:46.774362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 606/1197] 10.222.0.81 () {56 vars in 1177 bytes} [Thu Jun 16 18:39:46 2022] GET /identity/v3/users/3d3b100de68443f28966bb11a85ca0d3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:46.779987 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551ce14-fc7a-4e99-9f8e-54da3131ce46 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:46.875086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551ce14-fc7a-4e99-9f8e-54da3131ce46 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:46.907496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f61637-4f47-44e4-9c2c-20444b8f5927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:46.908206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f61637-4f47-44e4-9c2c-20444b8f5927 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:46.908490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f61637-4f47-44e4-9c2c-20444b8f5927 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:46.908769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f61637-4f47-44e4-9c2c-20444b8f5927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:46.967032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:46.973843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f61637-4f47-44e4-9c2c-20444b8f5927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:46.975274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f61637-4f47-44e4-9c2c-20444b8f5927 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:47.068494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 592/1198] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:39:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:39:47.074216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7eca9-c0b8-4279-a800-ee0c88d66deb None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:47.168126 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7eca9-c0b8-4279-a800-ee0c88d66deb None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:47.199572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b425a-bf4f-4d53-aae9-094a7ec0729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:47.200407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b425a-bf4f-4d53-aae9-094a7ec0729c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:47.200685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b425a-bf4f-4d53-aae9-094a7ec0729c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:47.200955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b425a-bf4f-4d53-aae9-094a7ec0729c None admin] PATH_INFO: `/v3/projects/b64ee1ed315e424e8963398f9560b109` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:47.202119 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b425a-bf4f-4d53-aae9-094a7ec0729c None admin] RBAC: Authorizing `identity:get_project(project_id=b64ee1ed315e424e8963398f9560b109)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:47.203529 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b425a-bf4f-4d53-aae9-094a7ec0729c None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:47.205753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 607/1199] 10.222.0.81 () {56 vars in 1186 bytes} [Thu Jun 16 18:39:47 2022] GET /identity/v3/projects/b64ee1ed315e424e8963398f9560b109 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:47.211705 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f61637-4f47-44e4-9c2c-20444b8f5927 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:39:47.300024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:47.305487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f61637-4f47-44e4-9c2c-20444b8f5927 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:39:47.337843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38472615-d48e-4c12-a855-218eb8bbf448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:39:47.338430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38472615-d48e-4c12-a855-218eb8bbf448 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:39:47.338717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38472615-d48e-4c12-a855-218eb8bbf448 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:39:47.338994 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38472615-d48e-4c12-a855-218eb8bbf448 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:39:47.339804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38472615-d48e-4c12-a855-218eb8bbf448 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=b64ee1ed315e424e8963398f9560b109, user.id=3d3b100de68443f28966bb11a85ca0d3) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:39:47.340170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38472615-d48e-4c12-a855-218eb8bbf448 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:39:47.341944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38472615-d48e-4c12-a855-218eb8bbf448 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:39:47.348075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 593/1200] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:39:47 2022] GET /identity/v3/role_assignments?user.id=3d3b100de68443f28966bb11a85ca0d3&scope.project.id=b64ee1ed315e424e8963398f9560b109&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:39:57.195632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=94 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:39:57.345828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=20 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:00.141191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc364cc-0fe9-4d83-b86a-c912af442187 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:00.141802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc364cc-0fe9-4d83-b86a-c912af442187 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:00.141802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc364cc-0fe9-4d83-b86a-c912af442187 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:00.143139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 608/1201] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:41:00.148159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae65112-d5ac-4f43-8da8-950b4cc72d4a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:00.148602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae65112-d5ac-4f43-8da8-950b4cc72d4a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:00.148710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae65112-d5ac-4f43-8da8-950b4cc72d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:00.165781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-bae65112-d5ac-4f43-8da8-950b4cc72d4a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:00.166962 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-bae65112-d5ac-4f43-8da8-950b4cc72d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:00.233329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 594/1202] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:41:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:41:00.257595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc364cc-0fe9-4d83-b86a-c912af442187 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:00.358851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc364cc-0fe9-4d83-b86a-c912af442187 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:00.392347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f5e84-bfa2-4c0f-ae5f-715d8abe4ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:00.392513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5e84-bfa2-4c0f-ae5f-715d8abe4ee4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:00.392733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5e84-bfa2-4c0f-ae5f-715d8abe4ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:00.393005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5e84-bfa2-4c0f-ae5f-715d8abe4ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:00.402720 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f5e84-bfa2-4c0f-ae5f-715d8abe4ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:00.404770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f5e84-bfa2-4c0f-ae5f-715d8abe4ee4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-cb1f5e84-bfa2-4c0f-ae5f-715d8abe4ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:00.421786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:00.428844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:00.430043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:00.430043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:00.430043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:00.430043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:00.430043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 609/1203] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:00.430355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae65112-d5ac-4f43-8da8-950b4cc72d4a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:00.526039 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae65112-d5ac-4f43-8da8-950b4cc72d4a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:00.554619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:00.560258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54305f0-20cf-4d6e-9ab4-a7b5fa92ccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:00.560917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54305f0-20cf-4d6e-9ab4-a7b5fa92ccd7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:00.561199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54305f0-20cf-4d6e-9ab4-a7b5fa92ccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:00.561507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54305f0-20cf-4d6e-9ab4-a7b5fa92ccd7 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:00.562210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54305f0-20cf-4d6e-9ab4-a7b5fa92ccd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:00.562523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54305f0-20cf-4d6e-9ab4-a7b5fa92ccd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:00.563891 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54305f0-20cf-4d6e-9ab4-a7b5fa92ccd7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:00.575697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 595/1204] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:00.581053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f5e84-bfa2-4c0f-ae5f-715d8abe4ee4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:00.632751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:00.677428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f5e84-bfa2-4c0f-ae5f-715d8abe4ee4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:00.709300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ff571-12e9-4849-b30a-11648404f079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:00.709959 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff571-12e9-4849-b30a-11648404f079 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:00.710240 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff571-12e9-4849-b30a-11648404f079 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:00.710512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff571-12e9-4849-b30a-11648404f079 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:00.711200 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ff571-12e9-4849-b30a-11648404f079 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:00.712609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ff571-12e9-4849-b30a-11648404f079 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:00.822101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 610/1205] 10.222.0.81 () {60 vars in 1132 bytes} [Thu Jun 16 18:41:00 2022] POST /identity/v3/users => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:41:02.968973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6cc26-b056-4159-a580-99b7411d2874 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:02.969561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6cc26-b056-4159-a580-99b7411d2874 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:02.969561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6cc26-b056-4159-a580-99b7411d2874 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:02.970986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 596/1206] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:41:02.976046 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e170f8-4127-4893-97dc-29e16b194995 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:02.976448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e170f8-4127-4893-97dc-29e16b194995 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:02.976613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e170f8-4127-4893-97dc-29e16b194995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:02.993826 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-86e170f8-4127-4893-97dc-29e16b194995 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:02.995013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-86e170f8-4127-4893-97dc-29e16b194995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:03.061281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 611/1207] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:41:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:41:03.085464 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6cc26-b056-4159-a580-99b7411d2874 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:03.185850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6cc26-b056-4159-a580-99b7411d2874 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:03.218409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13386cd7-ea76-4443-bd3c-ce5ebc289236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:03.219241 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13386cd7-ea76-4443-bd3c-ce5ebc289236 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:03.219474 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13386cd7-ea76-4443-bd3c-ce5ebc289236 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:03.219745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13386cd7-ea76-4443-bd3c-ce5ebc289236 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:03.229611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13386cd7-ea76-4443-bd3c-ce5ebc289236 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-13386cd7-ea76-4443-bd3c-ce5ebc289236 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:41:03.237825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:03.238955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:03.239862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 597/1208] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:41:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:03.245134 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e170f8-4127-4893-97dc-29e16b194995 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:03.283206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:03.339832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e170f8-4127-4893-97dc-29e16b194995 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:03.373571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14178daa-7951-41b8-850a-6aabb4e96ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:03.374234 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14178daa-7951-41b8-850a-6aabb4e96ca4 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:03.374519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14178daa-7951-41b8-850a-6aabb4e96ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:03.374801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14178daa-7951-41b8-850a-6aabb4e96ca4 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:03.375594 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14178daa-7951-41b8-850a-6aabb4e96ca4 None admin] RBAC: Adding query filter params (name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:03.376101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14178daa-7951-41b8-850a-6aabb4e96ca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:03.377849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14178daa-7951-41b8-850a-6aabb4e96ca4 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:03.386561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 612/1209] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:41:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:03.392388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13386cd7-ea76-4443-bd3c-ce5ebc289236 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:03.440962 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:03.490525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13386cd7-ea76-4443-bd3c-ce5ebc289236 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:03.523585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125912-13a4-4e47-8442-b167d9c7bc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:03.524370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125912-13a4-4e47-8442-b167d9c7bc21 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:03.524652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125912-13a4-4e47-8442-b167d9c7bc21 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:03.524923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee125912-13a4-4e47-8442-b167d9c7bc21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:03.535659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee125912-13a4-4e47-8442-b167d9c7bc21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:03.537210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee125912-13a4-4e47-8442-b167d9c7bc21 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-ee125912-13a4-4e47-8442-b167d9c7bc21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:03.553562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:03.554634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:03.555527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:03.555527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:03.555527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:03.555527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:03.555527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 598/1210] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:03.560755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14178daa-7951-41b8-850a-6aabb4e96ca4 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:03.658369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:03.661268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14178daa-7951-41b8-850a-6aabb4e96ca4 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:03.694092 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c044d2-a7ef-4b50-8298-f062c3c3a151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:03.694749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c044d2-a7ef-4b50-8298-f062c3c3a151 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:03.695036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c044d2-a7ef-4b50-8298-f062c3c3a151 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:03.695311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c044d2-a7ef-4b50-8298-f062c3c3a151 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:03.696025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c044d2-a7ef-4b50-8298-f062c3c3a151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:03.696334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c044d2-a7ef-4b50-8298-f062c3c3a151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:03.697746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c044d2-a7ef-4b50-8298-f062c3c3a151 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:03.709487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 613/1211] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:03.716166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125912-13a4-4e47-8442-b167d9c7bc21 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:03.815549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee125912-13a4-4e47-8442-b167d9c7bc21 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:03.817086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:03.848682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290cf3b7-2db8-4240-b18b-7de1ca4661a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:03.849509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cf3b7-2db8-4240-b18b-7de1ca4661a5 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:03.849793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cf3b7-2db8-4240-b18b-7de1ca4661a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:03.850070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cf3b7-2db8-4240-b18b-7de1ca4661a5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:03.856853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290cf3b7-2db8-4240-b18b-7de1ca4661a5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:03.858338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290cf3b7-2db8-4240-b18b-7de1ca4661a5 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-290cf3b7-2db8-4240-b18b-7de1ca4661a5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:41:03.873018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:03.874203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:03.875601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:03.875601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:03.875601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:41:03.875601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:41:03.875601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:41:03.875601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:41:03.875601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:41:03.875601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:41:03.875601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:03.875601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:03.875601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 599/1212] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:03.880077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c044d2-a7ef-4b50-8298-f062c3c3a151 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:03.979561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c044d2-a7ef-4b50-8298-f062c3c3a151 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:04.012504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd5f5c-8f7f-49ac-ba67-4fcf8dbbe622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:04.013160 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd5f5c-8f7f-49ac-ba67-4fcf8dbbe622 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:04.013481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd5f5c-8f7f-49ac-ba67-4fcf8dbbe622 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:04.013762 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd5f5c-8f7f-49ac-ba67-4fcf8dbbe622 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:04.014582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd5f5c-8f7f-49ac-ba67-4fcf8dbbe622 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:04.014893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd5f5c-8f7f-49ac-ba67-4fcf8dbbe622 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:04.016327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd5f5c-8f7f-49ac-ba67-4fcf8dbbe622 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:04.032862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 614/1213] 10.222.0.81 () {56 vars in 1143 bytes} [Thu Jun 16 18:41:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:04.038664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290cf3b7-2db8-4240-b18b-7de1ca4661a5 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:04.134498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290cf3b7-2db8-4240-b18b-7de1ca4661a5 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:04.167570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08a4d9-bc5e-4ccf-93f5-3e941629b400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:04.168204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08a4d9-bc5e-4ccf-93f5-3e941629b400 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:04.168489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08a4d9-bc5e-4ccf-93f5-3e941629b400 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:04.168764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08a4d9-bc5e-4ccf-93f5-3e941629b400 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:04.197502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:04.233716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08a4d9-bc5e-4ccf-93f5-3e941629b400 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:04.235201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08a4d9-bc5e-4ccf-93f5-3e941629b400 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:04.327370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 600/1214] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:41:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:41:04.333022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd5f5c-8f7f-49ac-ba67-4fcf8dbbe622 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:04.351979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:04.431421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd5f5c-8f7f-49ac-ba67-4fcf8dbbe622 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:04.465043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769300a-d513-45a3-90b0-c924ccc192e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:04.465852 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769300a-d513-45a3-90b0-c924ccc192e7 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:04.466134 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769300a-d513-45a3-90b0-c924ccc192e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:04.466406 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769300a-d513-45a3-90b0-c924ccc192e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:04.476480 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769300a-d513-45a3-90b0-c924ccc192e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:04.477979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769300a-d513-45a3-90b0-c924ccc192e7 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-6769300a-d513-45a3-90b0-c924ccc192e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:04.496166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:04.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:04.498158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:04.498158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:04.498158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:04.498158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:04.498158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 615/1215] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:04.503149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08a4d9-bc5e-4ccf-93f5-3e941629b400 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:04.561960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:04.599641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08a4d9-bc5e-4ccf-93f5-3e941629b400 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:04.631736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578995c-182e-48de-9f0f-475cec31449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:04.632411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578995c-182e-48de-9f0f-475cec31449d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:04.632679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578995c-182e-48de-9f0f-475cec31449d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:04.632953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578995c-182e-48de-9f0f-475cec31449d None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:04.633693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578995c-182e-48de-9f0f-475cec31449d None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:04.634002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578995c-182e-48de-9f0f-475cec31449d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:04.635363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578995c-182e-48de-9f0f-475cec31449d None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:04.647117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 601/1216] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:04.653172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769300a-d513-45a3-90b0-c924ccc192e7 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:04.720148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:04.749389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769300a-d513-45a3-90b0-c924ccc192e7 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:04.783476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35836cb5-0a58-453c-86b9-6be45032a980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:04.784280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35836cb5-0a58-453c-86b9-6be45032a980 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:04.784567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35836cb5-0a58-453c-86b9-6be45032a980 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:04.784846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35836cb5-0a58-453c-86b9-6be45032a980 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:04.792339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35836cb5-0a58-453c-86b9-6be45032a980 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:04.794127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35836cb5-0a58-453c-86b9-6be45032a980 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-35836cb5-0a58-453c-86b9-6be45032a980 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:41:04.808544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:04.809684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:04.811123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:41:04.811123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:41:04.811123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:41:04.811123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:41:04.811123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:41:04.811123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:41:04.811123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:04.811123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:04.811123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 616/1217] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:41:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:04.815766 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578995c-182e-48de-9f0f-475cec31449d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:04.912450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578995c-182e-48de-9f0f-475cec31449d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:04.933976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:04.945065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622c1f5-4b31-49ca-b880-d7fa63de01ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:04.945756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622c1f5-4b31-49ca-b880-d7fa63de01ff None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:04.946042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622c1f5-4b31-49ca-b880-d7fa63de01ff None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:04.946319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622c1f5-4b31-49ca-b880-d7fa63de01ff None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:04.947090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622c1f5-4b31-49ca-b880-d7fa63de01ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:04.948166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622c1f5-4b31-49ca-b880-d7fa63de01ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:04.949686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622c1f5-4b31-49ca-b880-d7fa63de01ff None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:04.960974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 602/1218] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:41:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:04.967689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35836cb5-0a58-453c-86b9-6be45032a980 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:05.065721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35836cb5-0a58-453c-86b9-6be45032a980 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:05.094590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:05.100374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7c865-130e-45ba-bd77-9390fd13d7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:05.101023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7c865-130e-45ba-bd77-9390fd13d7d1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:05.101339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7c865-130e-45ba-bd77-9390fd13d7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:05.103968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7c865-130e-45ba-bd77-9390fd13d7d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:05.103968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7c865-130e-45ba-bd77-9390fd13d7d1 None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=20c0fd976e684356b6143dc33fb7b4af) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:05.103968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7c865-130e-45ba-bd77-9390fd13d7d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:05.104264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd7c865-130e-45ba-bd77-9390fd13d7d1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:05.109571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 617/1219] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:41:04 2022] GET /identity/v3/role_assignments?user.id=20c0fd976e684356b6143dc33fb7b4af&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:41:07.201118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27dc4a-b82b-467d-8302-fd528a2b3657 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:07.201730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27dc4a-b82b-467d-8302-fd528a2b3657 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:07.201730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27dc4a-b82b-467d-8302-fd528a2b3657 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:07.203136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 603/1220] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:41:07.208470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a1039-fa51-41b6-889d-887299b85393 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:07.208894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a1039-fa51-41b6-889d-887299b85393 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:07.209032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a1039-fa51-41b6-889d-887299b85393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:07.225759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-588a1039-fa51-41b6-889d-887299b85393 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:07.226946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-588a1039-fa51-41b6-889d-887299b85393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:07.292690 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 618/1221] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:41:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 18:41:07.315295 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27dc4a-b82b-467d-8302-fd528a2b3657 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:07.414247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27dc4a-b82b-467d-8302-fd528a2b3657 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:07.447358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed7cc9-3607-49c4-8020-83ca13f5d20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:07.448139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed7cc9-3607-49c4-8020-83ca13f5d20d None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:07.448422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed7cc9-3607-49c4-8020-83ca13f5d20d None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:07.448695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed7cc9-3607-49c4-8020-83ca13f5d20d None admin] PATH_INFO: `/v3/roles/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:07.458673 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed7cc9-3607-49c4-8020-83ca13f5d20d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-84ed7cc9-3607-49c4-8020-83ca13f5d20d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:41:07.466530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:07.467756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:07.468868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 604/1222] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:41:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:07.474077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a1039-fa51-41b6-889d-887299b85393 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:07.569539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a1039-fa51-41b6-889d-887299b85393 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:07.592691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:07.603607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba5bb7-b1e1-4730-bca8-b26b6ff517d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:07.604270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba5bb7-b1e1-4730-bca8-b26b6ff517d8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:07.604556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba5bb7-b1e1-4730-bca8-b26b6ff517d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:07.604829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba5bb7-b1e1-4730-bca8-b26b6ff517d8 None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:07.605601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba5bb7-b1e1-4730-bca8-b26b6ff517d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:07.605913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba5bb7-b1e1-4730-bca8-b26b6ff517d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:07.607290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba5bb7-b1e1-4730-bca8-b26b6ff517d8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:07.615182 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 619/1223] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:41:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:07.620941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed7cc9-3607-49c4-8020-83ca13f5d20d None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:07.663948 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:07.717483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed7cc9-3607-49c4-8020-83ca13f5d20d None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:07.751080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c91de2-413d-469d-ab7d-118568e6612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:07.751962 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c91de2-413d-469d-ab7d-118568e6612e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:07.752345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c91de2-413d-469d-ab7d-118568e6612e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:07.752710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c91de2-413d-469d-ab7d-118568e6612e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:07.763363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c91de2-413d-469d-ab7d-118568e6612e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:07.764901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c91de2-413d-469d-ab7d-118568e6612e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-37c91de2-413d-469d-ab7d-118568e6612e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:07.782038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:07.783133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:07.784227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:07.784227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:07.784227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:07.784227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:07.784227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 605/1224] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:07.789097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba5bb7-b1e1-4730-bca8-b26b6ff517d8 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:07.886706 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba5bb7-b1e1-4730-bca8-b26b6ff517d8 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:07.919680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470e054-0d61-4d08-890a-3fa40efec41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:07.920340 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470e054-0d61-4d08-890a-3fa40efec41b None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:07.920623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470e054-0d61-4d08-890a-3fa40efec41b None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:07.920898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470e054-0d61-4d08-890a-3fa40efec41b None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:07.921632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470e054-0d61-4d08-890a-3fa40efec41b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:07.922103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470e054-0d61-4d08-890a-3fa40efec41b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:07.923495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470e054-0d61-4d08-890a-3fa40efec41b None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:07.934923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 620/1225] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:07.940584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c91de2-413d-469d-ab7d-118568e6612e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:08.033083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:08.039617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c91de2-413d-469d-ab7d-118568e6612e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:08.072431 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bfc0f-0b42-4357-b6d2-07c17039025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:08.073218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bfc0f-0b42-4357-b6d2-07c17039025f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:08.073536 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bfc0f-0b42-4357-b6d2-07c17039025f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:08.073817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bfc0f-0b42-4357-b6d2-07c17039025f None admin] PATH_INFO: `/v3/users/placement` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:08.080522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bfc0f-0b42-4357-b6d2-07c17039025f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:08.082020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bfc0f-0b42-4357-b6d2-07c17039025f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-5c8bfc0f-0b42-4357-b6d2-07c17039025f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:41:08.096853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:08.098017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:08.099210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:08.099210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:08.099210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:41:08.099210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:41:08.099210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:41:08.099210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:41:08.099210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:41:08.099210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:41:08.099210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:08.099210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:08.099210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 606/1226] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:08.104275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470e054-0d61-4d08-890a-3fa40efec41b None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:08.150374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:08.200903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470e054-0d61-4d08-890a-3fa40efec41b None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:08.234378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceba4ce-e40c-4965-8400-32f1114dc470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:08.235046 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceba4ce-e40c-4965-8400-32f1114dc470 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:08.235331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceba4ce-e40c-4965-8400-32f1114dc470 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:08.235607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceba4ce-e40c-4965-8400-32f1114dc470 None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:08.236422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceba4ce-e40c-4965-8400-32f1114dc470 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:08.236733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceba4ce-e40c-4965-8400-32f1114dc470 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:08.238188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceba4ce-e40c-4965-8400-32f1114dc470 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:08.257141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 621/1227] 10.222.0.81 () {56 vars in 1143 bytes} [Thu Jun 16 18:41:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:08.263639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bfc0f-0b42-4357-b6d2-07c17039025f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:08.358615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bfc0f-0b42-4357-b6d2-07c17039025f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:08.392262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3b712-e225-4728-9e8d-20d281408b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:08.393102 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3b712-e225-4728-9e8d-20d281408b8f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:08.393421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3b712-e225-4728-9e8d-20d281408b8f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:08.393700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3b712-e225-4728-9e8d-20d281408b8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:08.403301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3b712-e225-4728-9e8d-20d281408b8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:08.404825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc3b712-e225-4728-9e8d-20d281408b8f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-0dc3b712-e225-4728-9e8d-20d281408b8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:08.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:08.423047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:08.424190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:08.424190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:08.424190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:08.424190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:08.424190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 607/1228] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:41:08.428988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceba4ce-e40c-4965-8400-32f1114dc470 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:08.526656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceba4ce-e40c-4965-8400-32f1114dc470 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:08.528989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:08.560814 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669e619-25c1-4a54-b410-b4a9f4c4a1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:08.561558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669e619-25c1-4a54-b410-b4a9f4c4a1f3 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:08.561844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669e619-25c1-4a54-b410-b4a9f4c4a1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:08.562124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669e619-25c1-4a54-b410-b4a9f4c4a1f3 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:08.562840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669e619-25c1-4a54-b410-b4a9f4c4a1f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:08.563154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669e619-25c1-4a54-b410-b4a9f4c4a1f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:08.564521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669e619-25c1-4a54-b410-b4a9f4c4a1f3 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:08.576708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 622/1229] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:08.582464 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3b712-e225-4728-9e8d-20d281408b8f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:08.591119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:08.679734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc3b712-e225-4728-9e8d-20d281408b8f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:08.713097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c68a1b-6539-4d81-86fe-55dd7d128838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:08.713912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c68a1b-6539-4d81-86fe-55dd7d128838 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:08.714195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c68a1b-6539-4d81-86fe-55dd7d128838 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:08.714469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c68a1b-6539-4d81-86fe-55dd7d128838 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:08.720913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c68a1b-6539-4d81-86fe-55dd7d128838 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:08.722514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c68a1b-6539-4d81-86fe-55dd7d128838 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-03c68a1b-6539-4d81-86fe-55dd7d128838 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:41:08.736381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:08.737776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:08.738790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:41:08.738790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:41:08.738790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:41:08.738790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:41:08.738790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:41:08.738790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:41:08.738790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:08.738790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:08.738790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 608/1230] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:41:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:08.743436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669e619-25c1-4a54-b410-b4a9f4c4a1f3 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:08.842292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669e619-25c1-4a54-b410-b4a9f4c4a1f3 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:08.875684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ddbf6-dfa7-4315-aac6-c421157dd460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:08.876334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ddbf6-dfa7-4315-aac6-c421157dd460 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:08.876619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ddbf6-dfa7-4315-aac6-c421157dd460 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:08.876899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ddbf6-dfa7-4315-aac6-c421157dd460 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:08.877745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ddbf6-dfa7-4315-aac6-c421157dd460 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:08.878066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ddbf6-dfa7-4315-aac6-c421157dd460 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:08.879475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ddbf6-dfa7-4315-aac6-c421157dd460 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:08.891185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 623/1231] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:41:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:08.897089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c68a1b-6539-4d81-86fe-55dd7d128838 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:08.962083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:08.995636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c68a1b-6539-4d81-86fe-55dd7d128838 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:09.028581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb177a-4f53-4240-b8eb-ce8497fd4047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:09.029470 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb177a-4f53-4240-b8eb-ce8497fd4047 None admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:09.029755 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb177a-4f53-4240-b8eb-ce8497fd4047 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:09.030039 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb177a-4f53-4240-b8eb-ce8497fd4047 None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/20c0fd976e684356b6143dc33fb7b4af/roles/9718ec052fba437f923e8b15e08d7bed` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:09.044660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb177a-4f53-4240-b8eb-ce8497fd4047 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=20c0fd976e684356b6143dc33fb7b4af, role_id=9718ec052fba437f923e8b15e08d7bed)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:09.046126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb177a-4f53-4240-b8eb-ce8497fd4047 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:09.151579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 609/1232] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:41:08 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/20c0fd976e684356b6143dc33fb7b4af/roles/9718ec052fba437f923e8b15e08d7bed => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:41:11.251480 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b185e-843f-45ec-a1bc-1b11a31073e9 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:11.252070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b185e-843f-45ec-a1bc-1b11a31073e9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:11.252070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b185e-843f-45ec-a1bc-1b11a31073e9 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:11.253433 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 624/1233] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:41:11.259170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9431e-d898-4c43-89e6-fa4297605b01 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:11.259608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9431e-d898-4c43-89e6-fa4297605b01 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:11.259730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9431e-d898-4c43-89e6-fa4297605b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:11.275850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-c4b9431e-d898-4c43-89e6-fa4297605b01 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:11.277026 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c4b9431e-d898-4c43-89e6-fa4297605b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:11.341575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 610/1234] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:41:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:41:11.365556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b185e-843f-45ec-a1bc-1b11a31073e9 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:11.390988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:11.465106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b185e-843f-45ec-a1bc-1b11a31073e9 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:11.497567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d48f9-c788-4d93-8ddb-fbc19f3c673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:11.498347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d48f9-c788-4d93-8ddb-fbc19f3c673c None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:11.498628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d48f9-c788-4d93-8ddb-fbc19f3c673c None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:11.498899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d48f9-c788-4d93-8ddb-fbc19f3c673c None admin] PATH_INFO: `/v3/roles/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:11.512059 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d48f9-c788-4d93-8ddb-fbc19f3c673c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-9a1d48f9-c788-4d93-8ddb-fbc19f3c673c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:41:11.517077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:11.518255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:11.519156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 625/1235] 10.222.0.81 () {56 vars in 1102 bytes} [Thu Jun 16 18:41:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:11.524298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9431e-d898-4c43-89e6-fa4297605b01 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:11.568450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:11.618982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9431e-d898-4c43-89e6-fa4297605b01 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:11.651033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f855b1-8eee-474c-af5b-fcd7ea2d428f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:11.651694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f855b1-8eee-474c-af5b-fcd7ea2d428f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:11.651978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f855b1-8eee-474c-af5b-fcd7ea2d428f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:11.652255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f855b1-8eee-474c-af5b-fcd7ea2d428f None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:11.653005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f855b1-8eee-474c-af5b-fcd7ea2d428f None admin] RBAC: Adding query filter params (name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:11.653343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f855b1-8eee-474c-af5b-fcd7ea2d428f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:11.654711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f855b1-8eee-474c-af5b-fcd7ea2d428f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:11.662257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 611/1236] 10.222.0.81 () {56 vars in 1103 bytes} [Thu Jun 16 18:41:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:11.668531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d48f9-c788-4d93-8ddb-fbc19f3c673c None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:11.753167 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:11.768501 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d48f9-c788-4d93-8ddb-fbc19f3c673c None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:11.801322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8e42e-5ff5-470a-9cb1-3d6deea1ba9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:11.802104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8e42e-5ff5-470a-9cb1-3d6deea1ba9d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:11.802386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8e42e-5ff5-470a-9cb1-3d6deea1ba9d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:11.802661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8e42e-5ff5-470a-9cb1-3d6deea1ba9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:11.812904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8e42e-5ff5-470a-9cb1-3d6deea1ba9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:11.814379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8e42e-5ff5-470a-9cb1-3d6deea1ba9d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-f9d8e42e-5ff5-470a-9cb1-3d6deea1ba9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:11.831446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:11.832576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:11.833728 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:11.833728 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:11.833728 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:11.833728 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:11.833728 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 626/1237] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:11.838337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f855b1-8eee-474c-af5b-fcd7ea2d428f None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:11.937553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f855b1-8eee-474c-af5b-fcd7ea2d428f None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:11.946403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:11.970127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbebd4-c1e9-45c8-9354-ccc1580f3738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:11.970789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbebd4-c1e9-45c8-9354-ccc1580f3738 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:11.971113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbebd4-c1e9-45c8-9354-ccc1580f3738 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:11.971391 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbebd4-c1e9-45c8-9354-ccc1580f3738 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:11.972141 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbebd4-c1e9-45c8-9354-ccc1580f3738 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:11.972466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbebd4-c1e9-45c8-9354-ccc1580f3738 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:11.973861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbebd4-c1e9-45c8-9354-ccc1580f3738 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:11.985884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 612/1238] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:11.992220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8e42e-5ff5-470a-9cb1-3d6deea1ba9d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:12.088271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8e42e-5ff5-470a-9cb1-3d6deea1ba9d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:12.121277 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba63e8-5273-483e-8e5f-a6e822528800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:12.122068 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba63e8-5273-483e-8e5f-a6e822528800 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:12.122344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba63e8-5273-483e-8e5f-a6e822528800 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:12.122612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba63e8-5273-483e-8e5f-a6e822528800 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:12.125218 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:12.129879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba63e8-5273-483e-8e5f-a6e822528800 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:12.131301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba63e8-5273-483e-8e5f-a6e822528800 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-02ba63e8-5273-483e-8e5f-a6e822528800 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:41:12.145660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:12.146803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:12.148453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:12.148453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:12.148453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:41:12.148453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:41:12.148453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:41:12.148453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:41:12.148453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:41:12.148453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:41:12.148453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:12.148453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:12.148453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 627/1239] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:12.153202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbebd4-c1e9-45c8-9354-ccc1580f3738 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:12.254846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbebd4-c1e9-45c8-9354-ccc1580f3738 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:12.288664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe406b6-d5ab-4b1f-9008-458aec0f2a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:12.289362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe406b6-d5ab-4b1f-9008-458aec0f2a62 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:12.289649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe406b6-d5ab-4b1f-9008-458aec0f2a62 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:12.289927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe406b6-d5ab-4b1f-9008-458aec0f2a62 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:12.290744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe406b6-d5ab-4b1f-9008-458aec0f2a62 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:12.291053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe406b6-d5ab-4b1f-9008-458aec0f2a62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:12.292491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe406b6-d5ab-4b1f-9008-458aec0f2a62 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:12.310366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 613/1240] 10.222.0.81 () {56 vars in 1143 bytes} [Thu Jun 16 18:41:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:12.315832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba63e8-5273-483e-8e5f-a6e822528800 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:12.414601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba63e8-5273-483e-8e5f-a6e822528800 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:12.448334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958475d1-beba-4721-9e1a-4742fa7a5ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:12.448977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958475d1-beba-4721-9e1a-4742fa7a5ca1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:12.449294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958475d1-beba-4721-9e1a-4742fa7a5ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:12.449579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958475d1-beba-4721-9e1a-4742fa7a5ca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:12.508568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:12.514898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958475d1-beba-4721-9e1a-4742fa7a5ca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:12.516346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958475d1-beba-4721-9e1a-4742fa7a5ca1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:12.610588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 628/1241] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:41:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:41:12.616779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe406b6-d5ab-4b1f-9008-458aec0f2a62 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:12.643387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:12.713371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe406b6-d5ab-4b1f-9008-458aec0f2a62 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:12.746659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a7215-3e3a-4a5c-9c17-c841c18d4fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:12.747446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a7215-3e3a-4a5c-9c17-c841c18d4fce None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:12.747731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a7215-3e3a-4a5c-9c17-c841c18d4fce None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:12.748008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a7215-3e3a-4a5c-9c17-c841c18d4fce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:12.758382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a7215-3e3a-4a5c-9c17-c841c18d4fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:12.759879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a7215-3e3a-4a5c-9c17-c841c18d4fce None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-0f9a7215-3e3a-4a5c-9c17-c841c18d4fce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:12.777011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:12.778009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:12.778901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:12.778901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:12.778901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:12.778901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:12.778901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 614/1242] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:12.784049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958475d1-beba-4721-9e1a-4742fa7a5ca1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:12.903446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:12.909141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958475d1-beba-4721-9e1a-4742fa7a5ca1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:12.941933 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7916bf-bea6-4c8a-b7c8-a6118d4c12a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:12.942632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7916bf-bea6-4c8a-b7c8-a6118d4c12a0 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:12.942914 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7916bf-bea6-4c8a-b7c8-a6118d4c12a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:12.943185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7916bf-bea6-4c8a-b7c8-a6118d4c12a0 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:12.944876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7916bf-bea6-4c8a-b7c8-a6118d4c12a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:12.945188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7916bf-bea6-4c8a-b7c8-a6118d4c12a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:12.946569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7916bf-bea6-4c8a-b7c8-a6118d4c12a0 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:12.957523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 629/1243] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:12.963288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a7215-3e3a-4a5c-9c17-c841c18d4fce None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:13.040163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:13.061308 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a7215-3e3a-4a5c-9c17-c841c18d4fce None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:13.095120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73314b93-e613-4eaf-ad62-e18bcdf8a90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:13.095895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314b93-e613-4eaf-ad62-e18bcdf8a90e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:13.096177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314b93-e613-4eaf-ad62-e18bcdf8a90e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:13.096446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314b93-e613-4eaf-ad62-e18bcdf8a90e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:13.103684 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73314b93-e613-4eaf-ad62-e18bcdf8a90e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:13.105119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73314b93-e613-4eaf-ad62-e18bcdf8a90e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-73314b93-e613-4eaf-ad62-e18bcdf8a90e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:41:13.119420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:13.120425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:13.121393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:41:13.121393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:41:13.121393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:41:13.121393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:41:13.121393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:41:13.121393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:41:13.121393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:13.121393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:13.121393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 615/1244] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:41:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:13.126568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7916bf-bea6-4c8a-b7c8-a6118d4c12a0 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:13.241358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7916bf-bea6-4c8a-b7c8-a6118d4c12a0 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:13.277386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08829554-8697-449e-9e7c-baed28ee504a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:13.278151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08829554-8697-449e-9e7c-baed28ee504a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:13.278437 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08829554-8697-449e-9e7c-baed28ee504a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:13.278707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08829554-8697-449e-9e7c-baed28ee504a None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:13.279476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08829554-8697-449e-9e7c-baed28ee504a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:13.279780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08829554-8697-449e-9e7c-baed28ee504a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:13.281336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08829554-8697-449e-9e7c-baed28ee504a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:13.293527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 630/1245] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:41:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:13.299565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73314b93-e613-4eaf-ad62-e18bcdf8a90e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:13.399257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73314b93-e613-4eaf-ad62-e18bcdf8a90e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:13.432105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f6299-0e71-4427-847d-fef8f91f5849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:13.432758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f6299-0e71-4427-847d-fef8f91f5849 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:13.433045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f6299-0e71-4427-847d-fef8f91f5849 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:13.433352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f6299-0e71-4427-847d-fef8f91f5849 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:13.434174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f6299-0e71-4427-847d-fef8f91f5849 None admin] RBAC: Adding query filter params (role.id=9718ec052fba437f923e8b15e08d7bed, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=20c0fd976e684356b6143dc33fb7b4af) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:13.434490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f6299-0e71-4427-847d-fef8f91f5849 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:13.435928 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f6299-0e71-4427-847d-fef8f91f5849 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:13.441662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 616/1246] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:41:13 2022] GET /identity/v3/role_assignments?user.id=20c0fd976e684356b6143dc33fb7b4af&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=9718ec052fba437f923e8b15e08d7bed => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:15.587579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebca13-a964-49f1-a45a-8a54271fd263 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:15.588100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebca13-a964-49f1-a45a-8a54271fd263 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:15.588100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebca13-a964-49f1-a45a-8a54271fd263 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:15.589508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 631/1247] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:41:15.594607 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59db6b-70d6-4a11-817e-a3aa55aef2ce None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:15.595066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59db6b-70d6-4a11-817e-a3aa55aef2ce None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:15.595192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59db6b-70d6-4a11-817e-a3aa55aef2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:15.600172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:15.611700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ee59db6b-70d6-4a11-817e-a3aa55aef2ce None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:15.613012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ee59db6b-70d6-4a11-817e-a3aa55aef2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:15.675599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 617/1248] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:41:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:41:15.699614 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebca13-a964-49f1-a45a-8a54271fd263 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:15.717848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:15.798669 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebca13-a964-49f1-a45a-8a54271fd263 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:15.832355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd1b3d-4435-4cfa-9344-a1a94af803de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:15.833160 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd1b3d-4435-4cfa-9344-a1a94af803de None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:15.833468 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd1b3d-4435-4cfa-9344-a1a94af803de None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:15.833743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd1b3d-4435-4cfa-9344-a1a94af803de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:15.843491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd1b3d-4435-4cfa-9344-a1a94af803de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-b3cd1b3d-4435-4cfa-9344-a1a94af803de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:41:15.851157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:15.852264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:15.853165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 632/1249] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:41:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:15.858363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59db6b-70d6-4a11-817e-a3aa55aef2ce None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:15.954609 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59db6b-70d6-4a11-817e-a3aa55aef2ce None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:15.982804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:15.987655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d80094-4620-4c23-8651-2f4eaadc627e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:15.988317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d80094-4620-4c23-8651-2f4eaadc627e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:15.988600 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d80094-4620-4c23-8651-2f4eaadc627e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:15.988884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d80094-4620-4c23-8651-2f4eaadc627e None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:15.990181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d80094-4620-4c23-8651-2f4eaadc627e None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:15.990533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d80094-4620-4c23-8651-2f4eaadc627e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:15.991898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d80094-4620-4c23-8651-2f4eaadc627e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:15.999726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 618/1250] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:41:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:16.006367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd1b3d-4435-4cfa-9344-a1a94af803de None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:16.083966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:16.106682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd1b3d-4435-4cfa-9344-a1a94af803de None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:16.139481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524d5d2-b306-4608-8a02-dd50847c479d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:16.140262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524d5d2-b306-4608-8a02-dd50847c479d None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:16.140548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524d5d2-b306-4608-8a02-dd50847c479d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:16.140824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524d5d2-b306-4608-8a02-dd50847c479d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:16.150523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524d5d2-b306-4608-8a02-dd50847c479d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:16.152256 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524d5d2-b306-4608-8a02-dd50847c479d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-9524d5d2-b306-4608-8a02-dd50847c479d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:16.169566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:16.170455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:16.171364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:16.171364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:16.171364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:16.171364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:16.171364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 633/1251] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:41:16.176538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d80094-4620-4c23-8651-2f4eaadc627e None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:16.274108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d80094-4620-4c23-8651-2f4eaadc627e None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:16.307021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea062f-8605-4d7b-a6c9-8c1ee7875483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:16.307682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea062f-8605-4d7b-a6c9-8c1ee7875483 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:16.308062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea062f-8605-4d7b-a6c9-8c1ee7875483 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:16.308343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea062f-8605-4d7b-a6c9-8c1ee7875483 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:16.309045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea062f-8605-4d7b-a6c9-8c1ee7875483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:16.309383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea062f-8605-4d7b-a6c9-8c1ee7875483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:16.310746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea062f-8605-4d7b-a6c9-8c1ee7875483 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:16.322277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 619/1252] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:16.327975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524d5d2-b306-4608-8a02-dd50847c479d None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:16.425990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524d5d2-b306-4608-8a02-dd50847c479d None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:16.451661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:16.459651 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e75a04-6128-4a96-a055-fcca070b55bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:16.460421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75a04-6128-4a96-a055-fcca070b55bf None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:16.460701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75a04-6128-4a96-a055-fcca070b55bf None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:16.461033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e75a04-6128-4a96-a055-fcca070b55bf None admin] PATH_INFO: `/v3/users/placement` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:16.467637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e75a04-6128-4a96-a055-fcca070b55bf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:16.469048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e75a04-6128-4a96-a055-fcca070b55bf None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-d1e75a04-6128-4a96-a055-fcca070b55bf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:41:16.483729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:16.484673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:16.485698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:16.485698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:16.485698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:41:16.485698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:41:16.485698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:41:16.485698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:41:16.485698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:41:16.485698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:41:16.485698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:16.485698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:16.485698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 634/1253] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:16.491317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea062f-8605-4d7b-a6c9-8c1ee7875483 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:16.544376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:16.587894 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea062f-8605-4d7b-a6c9-8c1ee7875483 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:16.620188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22380a-8baa-43a4-8942-5cefee8dc2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:16.620847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22380a-8baa-43a4-8942-5cefee8dc2eb None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:16.621126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22380a-8baa-43a4-8942-5cefee8dc2eb None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:16.621425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa22380a-8baa-43a4-8942-5cefee8dc2eb None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:16.622233 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22380a-8baa-43a4-8942-5cefee8dc2eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:16.622535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22380a-8baa-43a4-8942-5cefee8dc2eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:16.624029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa22380a-8baa-43a4-8942-5cefee8dc2eb None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:16.641010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 620/1254] 10.222.0.81 () {56 vars in 1143 bytes} [Thu Jun 16 18:41:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:16.646746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e75a04-6128-4a96-a055-fcca070b55bf None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:16.743103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e75a04-6128-4a96-a055-fcca070b55bf None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:16.776523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87268158-6ffe-4104-ab0b-a8d75358a562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:16.777164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268158-6ffe-4104-ab0b-a8d75358a562 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:16.777481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268158-6ffe-4104-ab0b-a8d75358a562 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:16.777760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268158-6ffe-4104-ab0b-a8d75358a562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:16.832749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:16.844851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87268158-6ffe-4104-ab0b-a8d75358a562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:16.846335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87268158-6ffe-4104-ab0b-a8d75358a562 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:16.941591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 635/1255] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:41:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:41:16.947698 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22380a-8baa-43a4-8942-5cefee8dc2eb None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:17.046250 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa22380a-8baa-43a4-8942-5cefee8dc2eb None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:17.059609 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:17.080328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c9e98-9c4c-40bf-a125-133528efa1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:17.081155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c9e98-9c4c-40bf-a125-133528efa1ea None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:17.081488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c9e98-9c4c-40bf-a125-133528efa1ea None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:17.081787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c9e98-9c4c-40bf-a125-133528efa1ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:17.091629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c9e98-9c4c-40bf-a125-133528efa1ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:17.093033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c9e98-9c4c-40bf-a125-133528efa1ea None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-c00c9e98-9c4c-40bf-a125-133528efa1ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:17.110576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:17.111633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:17.112764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:17.112764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:17.112764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:17.112764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:17.112764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 621/1256] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:17.118222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87268158-6ffe-4104-ab0b-a8d75358a562 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:17.202803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:17.215248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87268158-6ffe-4104-ab0b-a8d75358a562 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:17.253443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea4bea-14a1-4e58-bc61-9959b76255f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:17.254581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea4bea-14a1-4e58-bc61-9959b76255f8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:17.255135 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea4bea-14a1-4e58-bc61-9959b76255f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:17.255642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea4bea-14a1-4e58-bc61-9959b76255f8 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:17.257005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea4bea-14a1-4e58-bc61-9959b76255f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:17.257608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea4bea-14a1-4e58-bc61-9959b76255f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:17.260171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea4bea-14a1-4e58-bc61-9959b76255f8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:17.273266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 636/1257] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:17.279254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c9e98-9c4c-40bf-a125-133528efa1ea None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:17.377114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c9e98-9c4c-40bf-a125-133528efa1ea None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:17.410469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d757ec-39a8-45b3-8678-d5cfd2af7553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:17.411829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d757ec-39a8-45b3-8678-d5cfd2af7553 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:17.412109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d757ec-39a8-45b3-8678-d5cfd2af7553 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:17.412380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d757ec-39a8-45b3-8678-d5cfd2af7553 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:17.418834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d757ec-39a8-45b3-8678-d5cfd2af7553 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:17.420270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d757ec-39a8-45b3-8678-d5cfd2af7553 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-f0d757ec-39a8-45b3-8678-d5cfd2af7553 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:41:17.433582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:17.434649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:17.435957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:41:17.435957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:41:17.435957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:41:17.435957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:41:17.435957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:41:17.435957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:41:17.435957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:17.435957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:17.435957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 622/1258] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:41:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:17.440576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea4bea-14a1-4e58-bc61-9959b76255f8 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:17.540818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea4bea-14a1-4e58-bc61-9959b76255f8 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:17.588511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c692d-3ec4-4677-9075-c05c2b2a6f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:17.589169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c692d-3ec4-4677-9075-c05c2b2a6f84 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:17.589479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c692d-3ec4-4677-9075-c05c2b2a6f84 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:17.589754 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c692d-3ec4-4677-9075-c05c2b2a6f84 None admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:17.590528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c692d-3ec4-4677-9075-c05c2b2a6f84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:17.590836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c692d-3ec4-4677-9075-c05c2b2a6f84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:17.592290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c692d-3ec4-4677-9075-c05c2b2a6f84 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:17.604618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 637/1259] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:41:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:17.610778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d757ec-39a8-45b3-8678-d5cfd2af7553 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:17.636368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:17.708261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d757ec-39a8-45b3-8678-d5cfd2af7553 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:17.741017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e7658-e47c-41b6-af4e-cedcc29043b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:17.741690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e7658-e47c-41b6-af4e-cedcc29043b7 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:17.741976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e7658-e47c-41b6-af4e-cedcc29043b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:17.742254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e7658-e47c-41b6-af4e-cedcc29043b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:17.743066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e7658-e47c-41b6-af4e-cedcc29043b7 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=20c0fd976e684356b6143dc33fb7b4af) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:17.743379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e7658-e47c-41b6-af4e-cedcc29043b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:17.744803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e7658-e47c-41b6-af4e-cedcc29043b7 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:17.750118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 623/1260] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:41:17 2022] GET /identity/v3/role_assignments?user.id=20c0fd976e684356b6143dc33fb7b4af&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:19.843821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d53fe-d2f7-4e65-8b9b-d9d454a3f63a None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:19.844365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d53fe-d2f7-4e65-8b9b-d9d454a3f63a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:19.844365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d53fe-d2f7-4e65-8b9b-d9d454a3f63a None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:19.845825 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 638/1261] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:41:19.851727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0a8cd-0b0f-44bd-a0de-4eef53efb25b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:19.852151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0a8cd-0b0f-44bd-a0de-4eef53efb25b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:19.852310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0a8cd-0b0f-44bd-a0de-4eef53efb25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:19.868990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-26d0a8cd-0b0f-44bd-a0de-4eef53efb25b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:19.870270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-26d0a8cd-0b0f-44bd-a0de-4eef53efb25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:19.936083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 624/1262] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:41:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:41:19.959209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d53fe-d2f7-4e65-8b9b-d9d454a3f63a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:19.970217 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:20.056968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d53fe-d2f7-4e65-8b9b-d9d454a3f63a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:20.088763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53d3f0-5d23-4fe5-8509-509ff1bba1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:20.089563 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53d3f0-5d23-4fe5-8509-509ff1bba1ab None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:20.089853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53d3f0-5d23-4fe5-8509-509ff1bba1ab None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:20.090130 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53d3f0-5d23-4fe5-8509-509ff1bba1ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:20.101611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53d3f0-5d23-4fe5-8509-509ff1bba1ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-4d53d3f0-5d23-4fe5-8509-509ff1bba1ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:41:20.108976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:20.112005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:20.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 639/1263] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:41:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:20.117002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0a8cd-0b0f-44bd-a0de-4eef53efb25b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:20.135168 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:20.212672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0a8cd-0b0f-44bd-a0de-4eef53efb25b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:20.246733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e0ece-b879-4b23-b0f3-156713ab8b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:20.247422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e0ece-b879-4b23-b0f3-156713ab8b62 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:20.247709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e0ece-b879-4b23-b0f3-156713ab8b62 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:20.247992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e0ece-b879-4b23-b0f3-156713ab8b62 None admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:20.248740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e0ece-b879-4b23-b0f3-156713ab8b62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:20.249047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e0ece-b879-4b23-b0f3-156713ab8b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:20.250485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773e0ece-b879-4b23-b0f3-156713ab8b62 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:20.258799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 625/1264] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:20.264486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53d3f0-5d23-4fe5-8509-509ff1bba1ab None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:20.331398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:20.364037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53d3f0-5d23-4fe5-8509-509ff1bba1ab None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:20.397102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c21faa-0901-4392-afb1-9e411388e150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:20.397917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c21faa-0901-4392-afb1-9e411388e150 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:20.398259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c21faa-0901-4392-afb1-9e411388e150 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:20.398533 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c21faa-0901-4392-afb1-9e411388e150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:20.408824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c21faa-0901-4392-afb1-9e411388e150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:20.410984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c21faa-0901-4392-afb1-9e411388e150 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-74c21faa-0901-4392-afb1-9e411388e150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:20.427572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:20.428558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:20.429632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:20.429632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:20.429632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:20.429632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:20.429632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 640/1265] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:20.434589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e0ece-b879-4b23-b0f3-156713ab8b62 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:20.512997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:20.531519 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e0ece-b879-4b23-b0f3-156713ab8b62 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:20.564120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dc139-4495-4c47-a872-c695e706acb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:20.564775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dc139-4495-4c47-a872-c695e706acb5 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:20.565064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dc139-4495-4c47-a872-c695e706acb5 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:20.565379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dc139-4495-4c47-a872-c695e706acb5 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:20.566413 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989dc139-4495-4c47-a872-c695e706acb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:20.566722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989dc139-4495-4c47-a872-c695e706acb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:20.568097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989dc139-4495-4c47-a872-c695e706acb5 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:20.578937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 626/1266] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:20.585218 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c21faa-0901-4392-afb1-9e411388e150 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:20.681559 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c21faa-0901-4392-afb1-9e411388e150 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:20.701229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:20.715161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d57e3a-463b-4209-8ce0-fd866872b72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:20.715949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d57e3a-463b-4209-8ce0-fd866872b72a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:20.716236 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d57e3a-463b-4209-8ce0-fd866872b72a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:20.716508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d57e3a-463b-4209-8ce0-fd866872b72a None admin] PATH_INFO: `/v3/users/placement` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:20.723731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d57e3a-463b-4209-8ce0-fd866872b72a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:20.726140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d57e3a-463b-4209-8ce0-fd866872b72a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-05d57e3a-463b-4209-8ce0-fd866872b72a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:41:20.740510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:20.741867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:20.743096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:20.743096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:20.743096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:41:20.743096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:41:20.743096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:41:20.743096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:41:20.743096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:41:20.743096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:41:20.743096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:20.743096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:20.743096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 641/1267] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:20.746363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dc139-4495-4c47-a872-c695e706acb5 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:20.845224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dc139-4495-4c47-a872-c695e706acb5 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:20.876895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99953822-0715-4e95-b23e-b7bfb5249369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:20.877581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99953822-0715-4e95-b23e-b7bfb5249369 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:20.877860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99953822-0715-4e95-b23e-b7bfb5249369 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:20.878128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99953822-0715-4e95-b23e-b7bfb5249369 None admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:20.879437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99953822-0715-4e95-b23e-b7bfb5249369 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:20.879741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99953822-0715-4e95-b23e-b7bfb5249369 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:20.881162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99953822-0715-4e95-b23e-b7bfb5249369 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:20.890024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:20.899856 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 627/1268] 10.222.0.81 () {56 vars in 1143 bytes} [Thu Jun 16 18:41:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:20.905490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d57e3a-463b-4209-8ce0-fd866872b72a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:21.004513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d57e3a-463b-4209-8ce0-fd866872b72a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:21.036245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0480de4-7c92-476c-a7d5-c076abc372c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:21.037020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0480de4-7c92-476c-a7d5-c076abc372c1 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:21.037329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0480de4-7c92-476c-a7d5-c076abc372c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:21.037606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0480de4-7c92-476c-a7d5-c076abc372c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:21.047894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0480de4-7c92-476c-a7d5-c076abc372c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:21.049368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0480de4-7c92-476c-a7d5-c076abc372c1 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-d0480de4-7c92-476c-a7d5-c076abc372c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:21.067263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:21.068358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:21.071469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:21.071469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:21.071469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:21.071469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:21.071469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 642/1269] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:21.074317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99953822-0715-4e95-b23e-b7bfb5249369 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:21.172868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99953822-0715-4e95-b23e-b7bfb5249369 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:21.206667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229eaf8-f750-49e2-b353-0f21cdd89b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:21.207325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229eaf8-f750-49e2-b353-0f21cdd89b28 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:21.207628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229eaf8-f750-49e2-b353-0f21cdd89b28 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:21.207891 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229eaf8-f750-49e2-b353-0f21cdd89b28 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:21.208593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229eaf8-f750-49e2-b353-0f21cdd89b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:21.208901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229eaf8-f750-49e2-b353-0f21cdd89b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:21.210318 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229eaf8-f750-49e2-b353-0f21cdd89b28 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:21.221070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 628/1270] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:21.226854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0480de4-7c92-476c-a7d5-c076abc372c1 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:21.259106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:21.335132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0480de4-7c92-476c-a7d5-c076abc372c1 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:21.371616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab64e74-0fed-4f85-93e4-3f67d843c2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:21.372414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab64e74-0fed-4f85-93e4-3f67d843c2fa None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:21.372702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab64e74-0fed-4f85-93e4-3f67d843c2fa None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:21.373012 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab64e74-0fed-4f85-93e4-3f67d843c2fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:21.380559 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab64e74-0fed-4f85-93e4-3f67d843c2fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:21.382128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab64e74-0fed-4f85-93e4-3f67d843c2fa None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-1ab64e74-0fed-4f85-93e4-3f67d843c2fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:41:21.395802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:21.396935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:21.398080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:41:21.398080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:41:21.398080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:41:21.398080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:41:21.398080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:41:21.398080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:41:21.398080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:21.398080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:21.398080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 643/1271] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:41:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:21.402959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229eaf8-f750-49e2-b353-0f21cdd89b28 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:21.460411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:21.500491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229eaf8-f750-49e2-b353-0f21cdd89b28 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:21.531728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505c943-b465-47ed-ad8a-95ff27fc7ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:21.532414 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505c943-b465-47ed-ad8a-95ff27fc7ece None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:21.532749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505c943-b465-47ed-ad8a-95ff27fc7ece None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:21.533027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505c943-b465-47ed-ad8a-95ff27fc7ece None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:21.533834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f505c943-b465-47ed-ad8a-95ff27fc7ece None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:21.534147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f505c943-b465-47ed-ad8a-95ff27fc7ece None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:21.536090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f505c943-b465-47ed-ad8a-95ff27fc7ece None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:21.547747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 629/1272] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:41:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:21.554344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab64e74-0fed-4f85-93e4-3f67d843c2fa None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:21.643671 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:21.652025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab64e74-0fed-4f85-93e4-3f67d843c2fa None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:21.686695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964b788-842f-4f8f-913c-c7a11a91e67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:21.687596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b788-842f-4f8f-913c-c7a11a91e67d None admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:21.687860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b788-842f-4f8f-913c-c7a11a91e67d None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:21.688139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964b788-842f-4f8f-913c-c7a11a91e67d None admin] PATH_INFO: `/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/20c0fd976e684356b6143dc33fb7b4af/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:21.690109 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b788-842f-4f8f-913c-c7a11a91e67d None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7e521e91b4468dbe5b02121e21c8c1, user_id=20c0fd976e684356b6143dc33fb7b4af, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:21.691808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964b788-842f-4f8f-913c-c7a11a91e67d None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:21.772418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 644/1273] 10.222.0.81 () {58 vars in 1439 bytes} [Thu Jun 16 18:41:21 2022] PUT /identity/v3/projects/dc7e521e91b4468dbe5b02121e21c8c1/users/20c0fd976e684356b6143dc33fb7b4af/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:41:23.877359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8f4fd-ea96-4b01-bf88-15afc7b768cb None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:23.877919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8f4fd-ea96-4b01-bf88-15afc7b768cb None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:23.877919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8f4fd-ea96-4b01-bf88-15afc7b768cb None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:23.879252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 630/1274] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:41:23.884578 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a7a21-1c85-4ed7-86c9-0922ea6a20c6 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:23.884964 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a7a21-1c85-4ed7-86c9-0922ea6a20c6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:23.885141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a7a21-1c85-4ed7-86c9-0922ea6a20c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:23.901426 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-138a7a21-1c85-4ed7-86c9-0922ea6a20c6 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:23.902627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-138a7a21-1c85-4ed7-86c9-0922ea6a20c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:23.968014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 645/1275] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:41:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:41:23.992201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8f4fd-ea96-4b01-bf88-15afc7b768cb None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:24.091366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8f4fd-ea96-4b01-bf88-15afc7b768cb None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:24.108654 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:24.124842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24db88b-b94e-42b4-a5ed-532259308976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:24.125666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24db88b-b94e-42b4-a5ed-532259308976 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:24.125952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24db88b-b94e-42b4-a5ed-532259308976 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:24.126238 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24db88b-b94e-42b4-a5ed-532259308976 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:24.136074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24db88b-b94e-42b4-a5ed-532259308976 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-d24db88b-b94e-42b4-a5ed-532259308976 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 18:41:24.144089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:24.145201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:24.146437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 631/1276] 10.222.0.81 () {56 vars in 1096 bytes} [Thu Jun 16 18:41:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:24.151226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a7a21-1c85-4ed7-86c9-0922ea6a20c6 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:24.230202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:24.247240 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a7a21-1c85-4ed7-86c9-0922ea6a20c6 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:24.280741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36599914-3f68-486f-8743-c1283784a44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:24.281427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36599914-3f68-486f-8743-c1283784a44a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:24.281707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36599914-3f68-486f-8743-c1283784a44a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:24.281977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36599914-3f68-486f-8743-c1283784a44a None admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:24.282700 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36599914-3f68-486f-8743-c1283784a44a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:24.283003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36599914-3f68-486f-8743-c1283784a44a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:24.284901 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36599914-3f68-486f-8743-c1283784a44a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:24.292826 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 646/1277] 10.222.0.81 () {56 vars in 1099 bytes} [Thu Jun 16 18:41:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:24.298374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24db88b-b94e-42b4-a5ed-532259308976 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:24.398078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24db88b-b94e-42b4-a5ed-532259308976 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:24.431269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b9cbe-db65-40c2-bceb-a7af9aedc1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:24.432053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b9cbe-db65-40c2-bceb-a7af9aedc1dd None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:24.432341 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b9cbe-db65-40c2-bceb-a7af9aedc1dd None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:24.432622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b9cbe-db65-40c2-bceb-a7af9aedc1dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:24.443018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b9cbe-db65-40c2-bceb-a7af9aedc1dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:24.444470 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03b9cbe-db65-40c2-bceb-a7af9aedc1dd None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-b03b9cbe-db65-40c2-bceb-a7af9aedc1dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:24.460975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:24.462071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:24.463329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:24.463329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:24.463329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:24.463329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:24.463329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 632/1278] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:24.468073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36599914-3f68-486f-8743-c1283784a44a None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:24.565010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36599914-3f68-486f-8743-c1283784a44a None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:24.597998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feabd87-d6e8-4ac5-b68b-d80a92138c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:24.598656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feabd87-d6e8-4ac5-b68b-d80a92138c15 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:24.598941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feabd87-d6e8-4ac5-b68b-d80a92138c15 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:24.599226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feabd87-d6e8-4ac5-b68b-d80a92138c15 None admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:24.599936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feabd87-d6e8-4ac5-b68b-d80a92138c15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:24.600243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feabd87-d6e8-4ac5-b68b-d80a92138c15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:24.601631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feabd87-d6e8-4ac5-b68b-d80a92138c15 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:24.613655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 647/1279] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:24.619291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b9cbe-db65-40c2-bceb-a7af9aedc1dd None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:24.648839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:24.716764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b9cbe-db65-40c2-bceb-a7af9aedc1dd None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:24.750230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc579802-cb25-4b2c-9019-7f0f1dc31b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:24.751014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc579802-cb25-4b2c-9019-7f0f1dc31b09 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:24.751386 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc579802-cb25-4b2c-9019-7f0f1dc31b09 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:24.751670 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc579802-cb25-4b2c-9019-7f0f1dc31b09 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:24.758445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc579802-cb25-4b2c-9019-7f0f1dc31b09 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:24.759878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc579802-cb25-4b2c-9019-7f0f1dc31b09 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-dc579802-cb25-4b2c-9019-7f0f1dc31b09 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 18:41:24.774788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:24.775945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:24.777119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:24.777119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:24.777119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 18:41:24.777119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 18:41:24.777119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 18:41:24.777119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 18:41:24.777119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 18:41:24.777119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 18:41:24.777119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 18:41:24.777119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:24.777119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 633/1280] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:24.781940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feabd87-d6e8-4ac5-b68b-d80a92138c15 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:24.786275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:24.878319 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feabd87-d6e8-4ac5-b68b-d80a92138c15 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:24.913669 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdc579-983c-4dc6-a32a-3cbc5a383cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:24.914427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdc579-983c-4dc6-a32a-3cbc5a383cbf None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:24.914546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdc579-983c-4dc6-a32a-3cbc5a383cbf None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:24.914823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdc579-983c-4dc6-a32a-3cbc5a383cbf None admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:24.915649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdc579-983c-4dc6-a32a-3cbc5a383cbf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:24.915959 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdc579-983c-4dc6-a32a-3cbc5a383cbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:24.917414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdc579-983c-4dc6-a32a-3cbc5a383cbf None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:24.934287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 648/1281] 10.222.0.81 () {56 vars in 1143 bytes} [Thu Jun 16 18:41:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:24.940607 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc579802-cb25-4b2c-9019-7f0f1dc31b09 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:25.030036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:25.040018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc579802-cb25-4b2c-9019-7f0f1dc31b09 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:25.072808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2b32c-b353-4976-b30c-8b7029de8395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:25.073518 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2b32c-b353-4976-b30c-8b7029de8395 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:25.073805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2b32c-b353-4976-b30c-8b7029de8395 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:25.074078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2b32c-b353-4976-b30c-8b7029de8395 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:25.140265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2b32c-b353-4976-b30c-8b7029de8395 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:25.141853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2b32c-b353-4976-b30c-8b7029de8395 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:25.266067 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:25.276343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 634/1282] 10.222.0.81 () {58 vars in 1282 bytes} [Thu Jun 16 18:41:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 18:41:25.283593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdc579-983c-4dc6-a32a-3cbc5a383cbf None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:25.351341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:25.392214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdc579-983c-4dc6-a32a-3cbc5a383cbf None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:25.428254 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40404acb-af37-415e-92a4-a5d013e403cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:25.429072 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40404acb-af37-415e-92a4-a5d013e403cc None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:25.429394 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40404acb-af37-415e-92a4-a5d013e403cc None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:25.429671 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40404acb-af37-415e-92a4-a5d013e403cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:25.441419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40404acb-af37-415e-92a4-a5d013e403cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:25.442898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40404acb-af37-415e-92a4-a5d013e403cc None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-40404acb-af37-415e-92a4-a5d013e403cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 18:41:25.461203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:25.462302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:25.463229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 18:41:25.463229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 18:41:25.463229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 18:41:25.463229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:25.463229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 649/1283] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:25.468604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2b32c-b353-4976-b30c-8b7029de8395 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:25.573597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2b32c-b353-4976-b30c-8b7029de8395 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:25.608232 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431b39c-5ce7-4f63-91d3-42ca8d938418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:25.608898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431b39c-5ce7-4f63-91d3-42ca8d938418 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:25.609181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431b39c-5ce7-4f63-91d3-42ca8d938418 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:25.609484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431b39c-5ce7-4f63-91d3-42ca8d938418 None admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:25.610192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431b39c-5ce7-4f63-91d3-42ca8d938418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:25.610496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431b39c-5ce7-4f63-91d3-42ca8d938418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:25.611900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431b39c-5ce7-4f63-91d3-42ca8d938418 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:25.623202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 635/1284] 10.222.0.81 () {56 vars in 1109 bytes} [Thu Jun 16 18:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:41:25.629379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40404acb-af37-415e-92a4-a5d013e403cc None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:25.726859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40404acb-af37-415e-92a4-a5d013e403cc None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:25.739833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:25.758929 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f63380-bf9c-4012-8194-62e1589778e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:25.759717 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f63380-bf9c-4012-8194-62e1589778e8 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:25.760002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f63380-bf9c-4012-8194-62e1589778e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:25.760282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f63380-bf9c-4012-8194-62e1589778e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:25.766797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f63380-bf9c-4012-8194-62e1589778e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:25.768747 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f63380-bf9c-4012-8194-62e1589778e8 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application [None req-89f63380-bf9c-4012-8194-62e1589778e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 18:41:25.784274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:25.785352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:25.786704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 18:41:25.786704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 18:41:25.786704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 18:41:25.786704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 18:41:25.786704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 18:41:25.786704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 18:41:25.786704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 18:41:25.786704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: ERROR keystone.server.flask.application Jun 16 18:41:25.786704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 650/1285] 10.222.0.81 () {56 vars in 1111 bytes} [Thu Jun 16 18:41:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:25.791366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431b39c-5ce7-4f63-91d3-42ca8d938418 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:25.839772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:25.887780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431b39c-5ce7-4f63-91d3-42ca8d938418 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:25.920241 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56a254-1991-4006-924f-f482a898e953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:25.920890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56a254-1991-4006-924f-f482a898e953 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:25.921169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56a254-1991-4006-924f-f482a898e953 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:25.921466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56a254-1991-4006-924f-f482a898e953 None admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:25.922225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56a254-1991-4006-924f-f482a898e953 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:25.922529 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56a254-1991-4006-924f-f482a898e953 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:25.924060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56a254-1991-4006-924f-f482a898e953 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:25.935605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 636/1286] 10.222.0.81 () {56 vars in 1148 bytes} [Thu Jun 16 18:41:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:25.942116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f63380-bf9c-4012-8194-62e1589778e8 None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:26.038740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f63380-bf9c-4012-8194-62e1589778e8 None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:26.072460 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54d816-6c4d-4d6b-8a95-03e835060c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:26.073105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54d816-6c4d-4d6b-8a95-03e835060c80 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:26.073418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54d816-6c4d-4d6b-8a95-03e835060c80 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:26.073696 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54d816-6c4d-4d6b-8a95-03e835060c80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:26.074512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54d816-6c4d-4d6b-8a95-03e835060c80 None admin] RBAC: Adding query filter params (role.id=7bdd42ee67ad4b74a7d9b9738787ac99, scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1, user.id=20c0fd976e684356b6143dc33fb7b4af) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:26.074823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54d816-6c4d-4d6b-8a95-03e835060c80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:26.076232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54d816-6c4d-4d6b-8a95-03e835060c80 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:26.082017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 651/1287] 10.222.0.81 () {56 vars in 1374 bytes} [Thu Jun 16 18:41:25 2022] GET /identity/v3/role_assignments?user.id=20c0fd976e684356b6143dc33fb7b4af&scope.project.id=dc7e521e91b4468dbe5b02121e21c8c1&role.id=7bdd42ee67ad4b74a7d9b9738787ac99 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:28.297710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60e2cc-4c15-48c1-b0c0-3c931c3b54b2 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:28.298237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60e2cc-4c15-48c1-b0c0-3c931c3b54b2 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:28.298237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60e2cc-4c15-48c1-b0c0-3c931c3b54b2 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:28.299617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 637/1288] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:41:28.304770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca64b-4443-496d-bd9a-53ebedb46dc6 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:28.305201 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca64b-4443-496d-bd9a-53ebedb46dc6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:28.305362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca64b-4443-496d-bd9a-53ebedb46dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:28.322177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3e7ca64b-4443-496d-bd9a-53ebedb46dc6 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:28.323352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3e7ca64b-4443-496d-bd9a-53ebedb46dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:28.352585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:28.389253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 652/1289] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:41:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:41:28.411843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60e2cc-4c15-48c1-b0c0-3c931c3b54b2 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:28.515261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60e2cc-4c15-48c1-b0c0-3c931c3b54b2 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:28.525504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:28.547965 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205eb70-7ee9-4ca0-991a-72a2c3537126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:28.548702 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205eb70-7ee9-4ca0-991a-72a2c3537126 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:28.548985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205eb70-7ee9-4ca0-991a-72a2c3537126 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:28.549278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205eb70-7ee9-4ca0-991a-72a2c3537126 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:28.549921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205eb70-7ee9-4ca0-991a-72a2c3537126 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:28.551294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205eb70-7ee9-4ca0-991a-72a2c3537126 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-8205eb70-7ee9-4ca0-991a-72a2c3537126 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:28.563773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:28.564991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:28.566050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 638/1290] 10.222.0.81 () {56 vars in 1117 bytes} [Thu Jun 16 18:41:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:28.570933 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ca64b-4443-496d-bd9a-53ebedb46dc6 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:28.666180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ca64b-4443-496d-bd9a-53ebedb46dc6 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:28.699438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88ebad-9330-451a-af98-fa69c87eec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:28.700095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88ebad-9330-451a-af98-fa69c87eec38 None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:28.700382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88ebad-9330-451a-af98-fa69c87eec38 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:28.700662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88ebad-9330-451a-af98-fa69c87eec38 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:28.701382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88ebad-9330-451a-af98-fa69c87eec38 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:28.701693 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88ebad-9330-451a-af98-fa69c87eec38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:28.703048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88ebad-9330-451a-af98-fa69c87eec38 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:28.707812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 653/1291] 10.222.0.81 () {56 vars in 1116 bytes} [Thu Jun 16 18:41:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:28.713608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205eb70-7ee9-4ca0-991a-72a2c3537126 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:28.831642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205eb70-7ee9-4ca0-991a-72a2c3537126 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:28.864519 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d723f89-4f29-40a6-8450-6757d6739f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:28.865171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d723f89-4f29-40a6-8450-6757d6739f2f None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:28.865480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d723f89-4f29-40a6-8450-6757d6739f2f None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:28.865755 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d723f89-4f29-40a6-8450-6757d6739f2f None admin] PATH_INFO: `/v3/services` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:28.866540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d723f89-4f29-40a6-8450-6757d6739f2f None admin] RBAC: Adding query filter params (type=placement) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:28.866850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d723f89-4f29-40a6-8450-6757d6739f2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:28.868219 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d723f89-4f29-40a6-8450-6757d6739f2f None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:28.872476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 639/1292] 10.222.0.81 () {56 vars in 1116 bytes} [Thu Jun 16 18:41:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:31.022870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e98f5a-f15f-428e-b99f-dfcfe711653d None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:31.023446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e98f5a-f15f-428e-b99f-dfcfe711653d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:31.023446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e98f5a-f15f-428e-b99f-dfcfe711653d None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:31.024772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 654/1293] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:41:31.029934 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e684f-fbc2-4247-8eea-99baaa5b492d None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:31.031016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e684f-fbc2-4247-8eea-99baaa5b492d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:31.031110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e684f-fbc2-4247-8eea-99baaa5b492d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:31.047401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b09e684f-fbc2-4247-8eea-99baaa5b492d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:31.048590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b09e684f-fbc2-4247-8eea-99baaa5b492d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:31.076933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:31.112594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 640/1294] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:41:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:41:31.135960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e98f5a-f15f-428e-b99f-dfcfe711653d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:31.166641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:31.234037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e98f5a-f15f-428e-b99f-dfcfe711653d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:31.267966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed8317-bef4-4b12-aa59-3a5fff06fb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:31.268618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed8317-bef4-4b12-aa59-3a5fff06fb95 None admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:31.268903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed8317-bef4-4b12-aa59-3a5fff06fb95 None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:31.269182 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed8317-bef4-4b12-aa59-3a5fff06fb95 None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:31.269914 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed8317-bef4-4b12-aa59-3a5fff06fb95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:31.271265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed8317-bef4-4b12-aa59-3a5fff06fb95 None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:31.335957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 655/1295] 10.222.0.81 () {60 vars in 1141 bytes} [Thu Jun 16 18:41:31 2022] POST /identity/v3/services => generated 246 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:41:33.434303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72ec46-475a-4bf0-a37b-7d435cae219c None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:33.434862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72ec46-475a-4bf0-a37b-7d435cae219c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:33.434862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72ec46-475a-4bf0-a37b-7d435cae219c None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:33.436523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 641/1296] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:41:33.441543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bd6f0-673a-486b-9275-7e48a91e1413 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:33.441941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bd6f0-673a-486b-9275-7e48a91e1413 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:33.442158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bd6f0-673a-486b-9275-7e48a91e1413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:33.458365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0d0bd6f0-673a-486b-9275-7e48a91e1413 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:33.459527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0d0bd6f0-673a-486b-9275-7e48a91e1413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:33.528425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 656/1297] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:41:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:41:33.551611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72ec46-475a-4bf0-a37b-7d435cae219c None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:33.650825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72ec46-475a-4bf0-a37b-7d435cae219c None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:33.683445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831f2ce-55f4-4dde-92fd-3744a53aa1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:33.684241 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f2ce-55f4-4dde-92fd-3744a53aa1e0 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:33.684518 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f2ce-55f4-4dde-92fd-3744a53aa1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:33.684793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831f2ce-55f4-4dde-92fd-3744a53aa1e0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:33.685472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831f2ce-55f4-4dde-92fd-3744a53aa1e0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:33.687596 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f831f2ce-55f4-4dde-92fd-3744a53aa1e0 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-f831f2ce-55f4-4dde-92fd-3744a53aa1e0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:33.699494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:33.700677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:33.702454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 642/1298] 10.222.0.81 () {56 vars in 1117 bytes} [Thu Jun 16 18:41:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 18:41:33.707298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bd6f0-673a-486b-9275-7e48a91e1413 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:33.725082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:33.802320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bd6f0-673a-486b-9275-7e48a91e1413 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:33.836812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfb94e-2615-413a-b818-4b7cb637d64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:33.837495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb94e-2615-413a-b818-4b7cb637d64e None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:33.837783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb94e-2615-413a-b818-4b7cb637d64e None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:33.838061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfb94e-2615-413a-b818-4b7cb637d64e None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:33.838760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfb94e-2615-413a-b818-4b7cb637d64e None admin] RBAC: Adding query filter params (name=placement) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:33.839076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfb94e-2615-413a-b818-4b7cb637d64e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:33.840437 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfb94e-2615-413a-b818-4b7cb637d64e None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:33.845485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 657/1299] 10.222.0.81 () {56 vars in 1116 bytes} [Thu Jun 16 18:41:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:33.851324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831f2ce-55f4-4dde-92fd-3744a53aa1e0 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:33.883013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:33.947349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831f2ce-55f4-4dde-92fd-3744a53aa1e0 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:33.979204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220bcd2-71f3-4e45-9515-3962c447f99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:33.979844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220bcd2-71f3-4e45-9515-3962c447f99e None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:33.980131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220bcd2-71f3-4e45-9515-3962c447f99e None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:33.980404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220bcd2-71f3-4e45-9515-3962c447f99e None admin] PATH_INFO: `/v3/endpoints` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:33.981169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220bcd2-71f3-4e45-9515-3962c447f99e None admin] RBAC: Adding query filter params (interface=public, service_id=a1d83b7af1d74e0faa887edd4ed6b496, region_id=RegionOne) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:33.981508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220bcd2-71f3-4e45-9515-3962c447f99e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:33.983383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c220bcd2-71f3-4e45-9515-3962c447f99e None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:33.989089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 643/1300] 10.222.0.81 () {56 vars in 1251 bytes} [Thu Jun 16 18:41:33 2022] GET /identity/v3/endpoints?service_id=a1d83b7af1d74e0faa887edd4ed6b496&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:33.994856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfb94e-2615-413a-b818-4b7cb637d64e None admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:34.073137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:34.090239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfb94e-2615-413a-b818-4b7cb637d64e None admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:34.123220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444b9f2-0b5b-4efc-a997-37e3c52b556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:34.123861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444b9f2-0b5b-4efc-a997-37e3c52b556a None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:34.124143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444b9f2-0b5b-4efc-a997-37e3c52b556a None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:34.124426 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444b9f2-0b5b-4efc-a997-37e3c52b556a None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:34.125077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444b9f2-0b5b-4efc-a997-37e3c52b556a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:34.126441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444b9f2-0b5b-4efc-a997-37e3c52b556a None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:34.132924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 658/1301] 10.222.0.81 () {56 vars in 1087 bytes} [Thu Jun 16 18:41:33 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:41:36.254544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e548d-da02-43af-a059-651fdff7bc82 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:36.255072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e548d-da02-43af-a059-651fdff7bc82 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:36.255189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e548d-da02-43af-a059-651fdff7bc82 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:36.256456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 644/1302] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:41:36.261737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218c0d9-8e11-4215-a393-f7297f876e0d None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:36.262143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218c0d9-8e11-4215-a393-f7297f876e0d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:36.262298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218c0d9-8e11-4215-a393-f7297f876e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:36.278565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-5218c0d9-8e11-4215-a393-f7297f876e0d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:41:36.279752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5218c0d9-8e11-4215-a393-f7297f876e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:41:36.345358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 659/1303] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:41:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 18:41:36.368156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e548d-da02-43af-a059-651fdff7bc82 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:36.460082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:36.467154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e548d-da02-43af-a059-651fdff7bc82 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:36.500318 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a786d55-0f52-4ce1-b263-b5aa8a904312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:36.501100 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a786d55-0f52-4ce1-b263-b5aa8a904312 None admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:36.501410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a786d55-0f52-4ce1-b263-b5aa8a904312 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:36.501791 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a786d55-0f52-4ce1-b263-b5aa8a904312 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:36.502340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a786d55-0f52-4ce1-b263-b5aa8a904312 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:36.503743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a786d55-0f52-4ce1-b263-b5aa8a904312 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application [None req-7a786d55-0f52-4ce1-b263-b5aa8a904312 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 18:41:36.515368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 18:41:36.516429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application with Lock( Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self._enter() Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application return self.creator() Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application created_value = creator( Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: ERROR keystone.server.flask.application Jun 16 18:41:36.517656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 645/1304] 10.222.0.81 () {56 vars in 1117 bytes} [Thu Jun 16 18:41:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 18:41:36.522463 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218c0d9-8e11-4215-a393-f7297f876e0d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:36.601787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:36.624183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218c0d9-8e11-4215-a393-f7297f876e0d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:36.656641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411453e-d483-48bb-ba1d-79f5341cadeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:36.657323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411453e-d483-48bb-ba1d-79f5341cadeb None admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:36.657612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411453e-d483-48bb-ba1d-79f5341cadeb None admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:36.657891 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411453e-d483-48bb-ba1d-79f5341cadeb None admin] PATH_INFO: `/v3/services` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:36.658585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411453e-d483-48bb-ba1d-79f5341cadeb None admin] RBAC: Adding query filter params (name=placement) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:41:36.658899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411453e-d483-48bb-ba1d-79f5341cadeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:36.660290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411453e-d483-48bb-ba1d-79f5341cadeb None admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:36.665296 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 660/1305] 10.222.0.81 () {56 vars in 1116 bytes} [Thu Jun 16 18:41:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:41:36.671902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a786d55-0f52-4ce1-b263-b5aa8a904312 None admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:41:36.769514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a786d55-0f52-4ce1-b263-b5aa8a904312 None admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:41:36.802359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f9de1-e4d8-482d-8bda-a21ef1960b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:41:36.803005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f9de1-e4d8-482d-8bda-a21ef1960b65 None admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:41:36.803289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f9de1-e4d8-482d-8bda-a21ef1960b65 None admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:41:36.803564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f9de1-e4d8-482d-8bda-a21ef1960b65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:41:36.804316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f9de1-e4d8-482d-8bda-a21ef1960b65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:41:36.805716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f9de1-e4d8-482d-8bda-a21ef1960b65 None admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:41:36.876829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 646/1306] 10.222.0.81 () {60 vars in 1144 bytes} [Thu Jun 16 18:41:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:41:46.675052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=30 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:41:46.872421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:39.676223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2837e-0ba9-4450-b433-f644c6774f4c None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:39.676747 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2837e-0ba9-4450-b433-f644c6774f4c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:39.676747 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2837e-0ba9-4450-b433-f644c6774f4c None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:39.685544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeae38-bd5b-48b6-96c3-ff3ef0f3e84a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:39.685544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeae38-bd5b-48b6-96c3-ff3ef0f3e84a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:39.685544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeae38-bd5b-48b6-96c3-ff3ef0f3e84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:39.687997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 661/1307] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:47:39 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:39.766867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-0afeae38-bd5b-48b6-96c3-ff3ef0f3e84a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:39.768255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0afeae38-bd5b-48b6-96c3-ff3ef0f3e84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:39.946880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 647/1308] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:47:40.007539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b00e3a-5643-49a7-ab33-ee9fed8fffcd None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:40.008249 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b00e3a-5643-49a7-ab33-ee9fed8fffcd None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:40.008766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b00e3a-5643-49a7-ab33-ee9fed8fffcd None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:40.014577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 662/1309] 10.222.0.81 () {54 vars in 964 bytes} [Thu Jun 16 18:47:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:40.026599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e6525-e39f-4ab4-9aa0-c1912d639ea5 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:40.027440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e6525-e39f-4ab4-9aa0-c1912d639ea5 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:40.028071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e6525-e39f-4ab4-9aa0-c1912d639ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:40.141224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-9a0e6525-e39f-4ab4-9aa0-c1912d639ea5 None None] MFA Rules not processed for user `f4e05d75cd1142e18d7518b78edd88fa`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:40.142523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-9a0e6525-e39f-4ab4-9aa0-c1912d639ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:40.164834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:40.225868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 648/1310] 10.222.0.81 () {58 vars in 1061 bytes} [Thu Jun 16 18:47:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:47:40.235106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b00e3a-5643-49a7-ab33-ee9fed8fffcd None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:40.236851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a5b00e3a-5643-49a7-ab33-ee9fed8fffcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:40.364585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b00e3a-5643-49a7-ab33-ee9fed8fffcd None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:40.444178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db038af-c588-41c8-b9aa-4058e4c2d0e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:40.444832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db038af-c588-41c8-b9aa-4058e4c2d0e2 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:40.445115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db038af-c588-41c8-b9aa-4058e4c2d0e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:40.445421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db038af-c588-41c8-b9aa-4058e4c2d0e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:40.447074 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8db038af-c588-41c8-b9aa-4058e4c2d0e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:40.504002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:40.558893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db038af-c588-41c8-b9aa-4058e4c2d0e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:40.560629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db038af-c588-41c8-b9aa-4058e4c2d0e2 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:40.581286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 663/1311] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:47:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:47:43.020058 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3fd16-9d9a-4a78-9888-d497316a6439 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:43.020617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3fd16-9d9a-4a78-9888-d497316a6439 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:43.020617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3fd16-9d9a-4a78-9888-d497316a6439 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:43.022301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 649/1312] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:43.027395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6c4a4-30e4-426b-8ce1-aa03e23caeb8 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:43.027837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6c4a4-30e4-426b-8ce1-aa03e23caeb8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:43.028065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6c4a4-30e4-426b-8ce1-aa03e23caeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:43.118261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-b4a6c4a4-30e4-426b-8ce1-aa03e23caeb8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:43.119715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b4a6c4a4-30e4-426b-8ce1-aa03e23caeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:43.241033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 664/1313] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:47:43.281001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3fd16-9d9a-4a78-9888-d497316a6439 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:43.282920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-70c3fd16-9d9a-4a78-9888-d497316a6439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:43.400817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3fd16-9d9a-4a78-9888-d497316a6439 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:43.475176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:43.477924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf4e56-a100-427d-a5f6-093dd58d6665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:43.478574 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf4e56-a100-427d-a5f6-093dd58d6665 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:43.478854 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf4e56-a100-427d-a5f6-093dd58d6665 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:43.479124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf4e56-a100-427d-a5f6-093dd58d6665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:43.481648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f9bf4e56-a100-427d-a5f6-093dd58d6665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:43.593180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf4e56-a100-427d-a5f6-093dd58d6665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:43.594807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf4e56-a100-427d-a5f6-093dd58d6665 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:43.613201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 650/1314] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:47:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:47:46.333713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3753ae-51ca-4a04-bbe3-6bb72ab7fc37 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:46.334396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3753ae-51ca-4a04-bbe3-6bb72ab7fc37 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:46.334396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3753ae-51ca-4a04-bbe3-6bb72ab7fc37 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:46.335680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 665/1315] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:46.344116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d710594-f548-40e6-b7a9-a0e66cd21858 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:46.344116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d710594-f548-40e6-b7a9-a0e66cd21858 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:46.344116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d710594-f548-40e6-b7a9-a0e66cd21858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:46.409257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-9d710594-f548-40e6-b7a9-a0e66cd21858 None None] MFA Rules not processed for user `cf1ee67d28b6436fa3ec26eeae38390a`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:46.410448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-9d710594-f548-40e6-b7a9-a0e66cd21858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:46.459471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:46.503195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 651/1316] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:47:46.542224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3753ae-51ca-4a04-bbe3-6bb72ab7fc37 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:46.543783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-2f3753ae-51ca-4a04-bbe3-6bb72ab7fc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:46.553394 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:46.645377 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3753ae-51ca-4a04-bbe3-6bb72ab7fc37 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:46.718544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9db6f-627b-4e27-86f7-fed58de01e4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:46.719188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9db6f-627b-4e27-86f7-fed58de01e4d service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:46.719475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9db6f-627b-4e27-86f7-fed58de01e4d service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:46.719751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9db6f-627b-4e27-86f7-fed58de01e4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:46.721431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-91b9db6f-627b-4e27-86f7-fed58de01e4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:46.781184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:46.826420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9db6f-627b-4e27-86f7-fed58de01e4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:46.827855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9db6f-627b-4e27-86f7-fed58de01e4d service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:46.845982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 666/1317] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:47:46 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:47:49.775963 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c837ca4-17ea-458e-90b9-6bbd6ca95ed4 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:49.776502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c837ca4-17ea-458e-90b9-6bbd6ca95ed4 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:49.776502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c837ca4-17ea-458e-90b9-6bbd6ca95ed4 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:49.777921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 652/1318] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:49.783060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d2be3-0ba3-49d9-8052-6ce220a60027 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:49.783498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d2be3-0ba3-49d9-8052-6ce220a60027 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:49.783600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d2be3-0ba3-49d9-8052-6ce220a60027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:49.845840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-1a8d2be3-0ba3-49d9-8052-6ce220a60027 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:49.847028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-1a8d2be3-0ba3-49d9-8052-6ce220a60027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:49.939175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 667/1319] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:47:49.977807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c837ca4-17ea-458e-90b9-6bbd6ca95ed4 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:49.979372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1c837ca4-17ea-458e-90b9-6bbd6ca95ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:50.080825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c837ca4-17ea-458e-90b9-6bbd6ca95ed4 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:50.151701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47d990-436a-4c29-9e26-1f7084094d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:50.152340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47d990-436a-4c29-9e26-1f7084094d6a service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:50.152618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47d990-436a-4c29-9e26-1f7084094d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:50.152888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47d990-436a-4c29-9e26-1f7084094d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:50.154553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-dd47d990-436a-4c29-9e26-1f7084094d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:50.161800 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:50.258283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47d990-436a-4c29-9e26-1f7084094d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:50.259638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47d990-436a-4c29-9e26-1f7084094d6a service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:50.277394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 653/1320] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:47:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:47:53.373526 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2c22-804d-4159-a31d-4e4c81482543 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:53.374017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2c22-804d-4159-a31d-4e4c81482543 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:53.374112 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c2c22-804d-4159-a31d-4e4c81482543 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:53.375460 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 668/1321] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:53.380793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655f8bd-c495-4f4d-868a-3aa23be20588 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:53.381256 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655f8bd-c495-4f4d-868a-3aa23be20588 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:53.381523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655f8bd-c495-4f4d-868a-3aa23be20588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:53.443759 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6655f8bd-c495-4f4d-868a-3aa23be20588 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:53.444959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6655f8bd-c495-4f4d-868a-3aa23be20588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:53.505126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:53.547501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 654/1322] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:47:53.584235 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c2c22-804d-4159-a31d-4e4c81482543 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:53.585842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-1e0c2c22-804d-4159-a31d-4e4c81482543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:53.606893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:53.694711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c2c22-804d-4159-a31d-4e4c81482543 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:53.767285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34aa56e6-3d9c-4359-9cee-cebeef800fca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:53.767931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa56e6-3d9c-4359-9cee-cebeef800fca service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:53.768214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa56e6-3d9c-4359-9cee-cebeef800fca service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:53.768489 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa56e6-3d9c-4359-9cee-cebeef800fca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:53.770155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-34aa56e6-3d9c-4359-9cee-cebeef800fca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:53.867377 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:53.904849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa56e6-3d9c-4359-9cee-cebeef800fca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:53.906223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34aa56e6-3d9c-4359-9cee-cebeef800fca service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:53.925292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 669/1323] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:47:56.114106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b27b5-b854-4e4b-bbb8-e4713217dd51 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:56.114672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b27b5-b854-4e4b-bbb8-e4713217dd51 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:56.114672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9b27b5-b854-4e4b-bbb8-e4713217dd51 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:56.116021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 655/1324] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:56.121365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbda1a-b985-4e37-a951-f11745ea153b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:56.121793 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbda1a-b985-4e37-a951-f11745ea153b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:56.121907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbda1a-b985-4e37-a951-f11745ea153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:56.186389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-7acbda1a-b985-4e37-a951-f11745ea153b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:56.187594 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7acbda1a-b985-4e37-a951-f11745ea153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:56.284172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 670/1325] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:47:56.319777 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b27b5-b854-4e4b-bbb8-e4713217dd51 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:56.321920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1e9b27b5-b854-4e4b-bbb8-e4713217dd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:56.429577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9b27b5-b854-4e4b-bbb8-e4713217dd51 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:56.505210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f6ca7-465e-4c4e-ad1b-b1a5b3bfd850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:56.505871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f6ca7-465e-4c4e-ad1b-b1a5b3bfd850 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:56.506151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f6ca7-465e-4c4e-ad1b-b1a5b3bfd850 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:56.506422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f6ca7-465e-4c4e-ad1b-b1a5b3bfd850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:56.508058 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-732f6ca7-465e-4c4e-ad1b-b1a5b3bfd850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:56.526697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:56.615722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f6ca7-465e-4c4e-ad1b-b1a5b3bfd850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:56.617171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f6ca7-465e-4c4e-ad1b-b1a5b3bfd850 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:56.635648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 656/1326] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:47:58.824472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb550d-c2b1-471c-8d73-2ac69b6b783b None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:58.824917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb550d-c2b1-471c-8d73-2ac69b6b783b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:58.825031 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb550d-c2b1-471c-8d73-2ac69b6b783b None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:58.826431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 671/1327] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:47:58.831775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c865e9b-24e3-4acf-95e2-2b395273d40e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:58.832183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c865e9b-24e3-4acf-95e2-2b395273d40e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:58.832334 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c865e9b-24e3-4acf-95e2-2b395273d40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:58.896405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1c865e9b-24e3-4acf-95e2-2b395273d40e None None] MFA Rules not processed for user `cf1ee67d28b6436fa3ec26eeae38390a`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:47:58.897662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1c865e9b-24e3-4acf-95e2-2b395273d40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:58.962053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:59.001180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 657/1328] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:47:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:47:59.031710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb550d-c2b1-471c-8d73-2ac69b6b783b None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:47:59.033297 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c8bb550d-c2b1-471c-8d73-2ac69b6b783b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:59.070806 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:59.155574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb550d-c2b1-471c-8d73-2ac69b6b783b None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:47:59.248824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b422d-8b19-479c-af22-829199964928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:47:59.249541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b422d-8b19-479c-af22-829199964928 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:47:59.249823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b422d-8b19-479c-af22-829199964928 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:47:59.250096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b422d-8b19-479c-af22-829199964928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:47:59.251738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-540b422d-8b19-479c-af22-829199964928 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:47:59.353467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:47:59.389099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b422d-8b19-479c-af22-829199964928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:47:59.390578 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b422d-8b19-479c-af22-829199964928 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:47:59.409404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 672/1329] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:47:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:02.002525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e7515-63cd-41c0-a054-00765c971caf None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:02.003042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e7515-63cd-41c0-a054-00765c971caf None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:02.003042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e7515-63cd-41c0-a054-00765c971caf None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:02.004435 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 658/1330] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:02.010264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed9e5c-497c-4578-b00f-67aea0c9e95c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:02.010701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed9e5c-497c-4578-b00f-67aea0c9e95c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:02.010868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed9e5c-497c-4578-b00f-67aea0c9e95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:02.074388 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-c0ed9e5c-497c-4578-b00f-67aea0c9e95c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:02.075618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c0ed9e5c-497c-4578-b00f-67aea0c9e95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:02.169626 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 673/1331] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:02.206177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e7515-63cd-41c0-a054-00765c971caf None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:02.207711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8c5e7515-63cd-41c0-a054-00765c971caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:02.310394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e7515-63cd-41c0-a054-00765c971caf None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:02.382283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0df27-0fa8-4d2b-8d68-cf98569b48f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:02.383211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0df27-0fa8-4d2b-8d68-cf98569b48f7 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:02.383502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0df27-0fa8-4d2b-8d68-cf98569b48f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:02.383776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0df27-0fa8-4d2b-8d68-cf98569b48f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:02.385453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c7f0df27-0fa8-4d2b-8d68-cf98569b48f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:02.411203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:02.491584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0df27-0fa8-4d2b-8d68-cf98569b48f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:02.493014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0df27-0fa8-4d2b-8d68-cf98569b48f7 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:02.511305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 659/1332] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:04.647850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd288be-8f85-4be2-bac1-df7f62055b10 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:04.648461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd288be-8f85-4be2-bac1-df7f62055b10 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:04.648461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd288be-8f85-4be2-bac1-df7f62055b10 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:04.649723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 674/1333] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:04.654712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005323a7-55c7-4556-9dbd-1de02c4dfddb None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:04.655136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005323a7-55c7-4556-9dbd-1de02c4dfddb None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:04.655305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005323a7-55c7-4556-9dbd-1de02c4dfddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:04.717886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-005323a7-55c7-4556-9dbd-1de02c4dfddb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:04.719080 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-005323a7-55c7-4556-9dbd-1de02c4dfddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:04.790252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:04.811566 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 660/1334] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:04.867485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37714a-b987-4a39-b44a-24f755b561fc None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:04.867773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37714a-b987-4a39-b44a-24f755b561fc None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:04.868049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37714a-b987-4a39-b44a-24f755b561fc None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:04.869455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 675/1335] 10.222.0.81 () {54 vars in 964 bytes} [Thu Jun 16 18:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:04.875168 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f087a2-37cb-4771-84d5-5c45d8d52ebb None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:04.875461 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f087a2-37cb-4771-84d5-5c45d8d52ebb None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:04.875739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f087a2-37cb-4771-84d5-5c45d8d52ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:04.949026 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-c2f087a2-37cb-4771-84d5-5c45d8d52ebb None None] MFA Rules not processed for user `f4e05d75cd1142e18d7518b78edd88fa`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:04.950190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c2f087a2-37cb-4771-84d5-5c45d8d52ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:05.023735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 661/1336] 10.222.0.81 () {58 vars in 1061 bytes} [Thu Jun 16 18:48:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:05.032615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37714a-b987-4a39-b44a-24f755b561fc None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:05.034151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-fa37714a-b987-4a39-b44a-24f755b561fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:05.073749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:05.137281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37714a-b987-4a39-b44a-24f755b561fc None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:05.208594 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb0c40-4f97-4441-826d-75061eb4fa16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:05.209168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb0c40-4f97-4441-826d-75061eb4fa16 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:05.209508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb0c40-4f97-4441-826d-75061eb4fa16 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:05.209791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb0c40-4f97-4441-826d-75061eb4fa16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:05.211423 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f7bb0c40-4f97-4441-826d-75061eb4fa16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:05.292732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:05.319405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb0c40-4f97-4441-826d-75061eb4fa16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:05.320838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb0c40-4f97-4441-826d-75061eb4fa16 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:05.338739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 676/1337] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:08.570263 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862075a-d708-43a2-b603-31b536066452 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:08.570794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862075a-d708-43a2-b603-31b536066452 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:08.570794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862075a-d708-43a2-b603-31b536066452 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:08.572175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 662/1338] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:08.577279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab470f-893c-483a-bdaf-c29a42700a92 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:08.577655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab470f-893c-483a-bdaf-c29a42700a92 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:08.577845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ab470f-893c-483a-bdaf-c29a42700a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:08.640902 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-40ab470f-893c-483a-bdaf-c29a42700a92 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:08.642084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-40ab470f-893c-483a-bdaf-c29a42700a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:08.735783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 677/1339] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:08.771807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862075a-d708-43a2-b603-31b536066452 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:08.773371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a862075a-d708-43a2-b603-31b536066452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:08.846291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:08.873155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862075a-d708-43a2-b603-31b536066452 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:08.947650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1595a8-96b3-42ad-bf6e-29a7ffcaadbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:08.948259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1595a8-96b3-42ad-bf6e-29a7ffcaadbe service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:08.948546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1595a8-96b3-42ad-bf6e-29a7ffcaadbe service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:08.948824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1595a8-96b3-42ad-bf6e-29a7ffcaadbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:08.950532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-dd1595a8-96b3-42ad-bf6e-29a7ffcaadbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:09.056885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1595a8-96b3-42ad-bf6e-29a7ffcaadbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:09.058374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1595a8-96b3-42ad-bf6e-29a7ffcaadbe service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:09.076143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 663/1340] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:13.599330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c16bb-e280-4b2f-af74-783dd4366357 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:13.599910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c16bb-e280-4b2f-af74-783dd4366357 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:13.599910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c16bb-e280-4b2f-af74-783dd4366357 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:13.601817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 678/1341] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:13.606740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae59a1-6f7f-4be2-b79e-ccc3f58ca963 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:13.607177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae59a1-6f7f-4be2-b79e-ccc3f58ca963 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:13.607280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae59a1-6f7f-4be2-b79e-ccc3f58ca963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:13.616495 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:13.688070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ebae59a1-6f7f-4be2-b79e-ccc3f58ca963 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:13.689491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ebae59a1-6f7f-4be2-b79e-ccc3f58ca963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:13.793584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 664/1342] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:13.832047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c16bb-e280-4b2f-af74-783dd4366357 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:13.834424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-651c16bb-e280-4b2f-af74-783dd4366357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:13.884793 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:13.930625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c16bb-e280-4b2f-af74-783dd4366357 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:13.934380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcb66a-5161-43aa-bd80-2f2f569bdcce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:13.935027 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcb66a-5161-43aa-bd80-2f2f569bdcce service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:13.935312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcb66a-5161-43aa-bd80-2f2f569bdcce service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:13.935607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcb66a-5161-43aa-bd80-2f2f569bdcce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:13.944640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcb66a-5161-43aa-bd80-2f2f569bdcce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:13.946115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcb66a-5161-43aa-bd80-2f2f569bdcce service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:13.955472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 679/1343] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:48:17.039922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d2dac-351d-4be2-8764-ffa37c5b409c None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:17.040538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d2dac-351d-4be2-8764-ffa37c5b409c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:17.040538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d2dac-351d-4be2-8764-ffa37c5b409c None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:17.041904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 665/1344] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:17.047598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51804623-3d92-4006-87b5-4823208020d4 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:17.048154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51804623-3d92-4006-87b5-4823208020d4 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:17.048154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51804623-3d92-4006-87b5-4823208020d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:17.076133 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-51804623-3d92-4006-87b5-4823208020d4 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:17.079892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-51804623-3d92-4006-87b5-4823208020d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:17.154376 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 680/1345] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:17.192091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d2dac-351d-4be2-8764-ffa37c5b409c None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:17.202709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d2dac-351d-4be2-8764-ffa37c5b409c None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:17.206139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cc57ad-a793-4500-aab6-11c090f956ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:17.206802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc57ad-a793-4500-aab6-11c090f956ce service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:17.207088 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc57ad-a793-4500-aab6-11c090f956ce service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:17.207365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cc57ad-a793-4500-aab6-11c090f956ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:17.216549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc57ad-a793-4500-aab6-11c090f956ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:17.218022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cc57ad-a793-4500-aab6-11c090f956ce service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:17.227445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 666/1346] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:21.489492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ff00d-f262-47c2-b274-16b1747f7de4 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:21.490057 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ff00d-f262-47c2-b274-16b1747f7de4 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:21.490057 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ff00d-f262-47c2-b274-16b1747f7de4 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:21.491407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 681/1347] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:21.496506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe5ee8-e362-4885-8a8d-301cd4b89128 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:21.496943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe5ee8-e362-4885-8a8d-301cd4b89128 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:21.497045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe5ee8-e362-4885-8a8d-301cd4b89128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:21.514674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-4dfe5ee8-e362-4885-8a8d-301cd4b89128 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:21.515859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-4dfe5ee8-e362-4885-8a8d-301cd4b89128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:21.524256 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 667/1348] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:21.561682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ff00d-f262-47c2-b274-16b1747f7de4 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:21.572933 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ff00d-f262-47c2-b274-16b1747f7de4 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:21.576482 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98f6a42-bca7-42b5-8e43-a88c0f919c19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:21.577123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f6a42-bca7-42b5-8e43-a88c0f919c19 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:21.577473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f6a42-bca7-42b5-8e43-a88c0f919c19 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:21.577752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98f6a42-bca7-42b5-8e43-a88c0f919c19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:21.587729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f6a42-bca7-42b5-8e43-a88c0f919c19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:21.589178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98f6a42-bca7-42b5-8e43-a88c0f919c19 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:21.599315 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 682/1349] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:26.609174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223a51a-af05-4894-b44d-3505da68ad01 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:26.609712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223a51a-af05-4894-b44d-3505da68ad01 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:26.609712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223a51a-af05-4894-b44d-3505da68ad01 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:26.611923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 668/1350] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 18:48:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:26.616739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd63f5b-be60-4413-9cec-4fd93bc3bc79 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:26.617127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd63f5b-be60-4413-9cec-4fd93bc3bc79 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:26.617439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd63f5b-be60-4413-9cec-4fd93bc3bc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:26.691360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3fd63f5b-be60-4413-9cec-4fd93bc3bc79 None None] MFA Rules not processed for user `20c0fd976e684356b6143dc33fb7b4af`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:26.693001 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3fd63f5b-be60-4413-9cec-4fd93bc3bc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:26.725375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:26.779896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 683/1351] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 18:48:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:31.021175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de5a13-85b2-4a3f-99d4-8dbd24b0c9af None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:31.021728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de5a13-85b2-4a3f-99d4-8dbd24b0c9af None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:31.021843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de5a13-85b2-4a3f-99d4-8dbd24b0c9af None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:31.023312 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 669/1352] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 18:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:31.028732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981f849-a7bb-48e3-a20f-17317f85311e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:31.029168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981f849-a7bb-48e3-a20f-17317f85311e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:31.029327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981f849-a7bb-48e3-a20f-17317f85311e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:31.047537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-5981f849-a7bb-48e3-a20f-17317f85311e None None] MFA Rules not processed for user `20c0fd976e684356b6143dc33fb7b4af`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:31.049015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5981f849-a7bb-48e3-a20f-17317f85311e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:31.058095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 684/1353] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 18:48:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:31.509639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=72 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:32.506337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6038c7-c163-4ee0-8ac9-47e17ab0d22d None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:32.506933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6038c7-c163-4ee0-8ac9-47e17ab0d22d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:32.506933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6038c7-c163-4ee0-8ac9-47e17ab0d22d None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:32.508227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 670/1354] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 18:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:32.514386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b3536-0ae0-45ba-b925-4520d618da46 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:32.514853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b3536-0ae0-45ba-b925-4520d618da46 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:32.514961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b3536-0ae0-45ba-b925-4520d618da46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:32.534499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-462b3536-0ae0-45ba-b925-4520d618da46 None None] MFA Rules not processed for user `20c0fd976e684356b6143dc33fb7b4af`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:32.535942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-462b3536-0ae0-45ba-b925-4520d618da46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:32.544565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 685/1355] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 18:48:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:36.442192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46997b5c-b0c8-4982-869d-f5b882a49904 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:36.443209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46997b5c-b0c8-4982-869d-f5b882a49904 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:36.443714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46997b5c-b0c8-4982-869d-f5b882a49904 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:36.445526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 671/1356] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:36.450503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5c3c-648b-4f10-a402-c6971c2cae66 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:36.450906 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5c3c-648b-4f10-a402-c6971c2cae66 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:36.451091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5c3c-648b-4f10-a402-c6971c2cae66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:36.470331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-a09e5c3c-648b-4f10-a402-c6971c2cae66 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:36.471447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a09e5c3c-648b-4f10-a402-c6971c2cae66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:36.479343 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 686/1357] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:36.534376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c5200-b658-47c6-b2d4-1972e255ab2f None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:36.535155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c5200-b658-47c6-b2d4-1972e255ab2f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:36.535619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c5200-b658-47c6-b2d4-1972e255ab2f None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:36.537354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 672/1358] 10.222.0.81 () {54 vars in 961 bytes} [Thu Jun 16 18:48:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:36.549101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47426c3e-d855-4a83-87f8-1ef4efee0a76 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:36.549653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47426c3e-d855-4a83-87f8-1ef4efee0a76 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:36.550220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47426c3e-d855-4a83-87f8-1ef4efee0a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:36.601911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-47426c3e-d855-4a83-87f8-1ef4efee0a76 None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:36.608210 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-47426c3e-d855-4a83-87f8-1ef4efee0a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:36.703227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 687/1359] 10.222.0.81 () {58 vars in 1058 bytes} [Thu Jun 16 18:48:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:36.718646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c5200-b658-47c6-b2d4-1972e255ab2f None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:36.737331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c5200-b658-47c6-b2d4-1972e255ab2f None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:36.743315 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6218bca-c440-4be0-a7a4-aa124245a1fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:36.744888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6218bca-c440-4be0-a7a4-aa124245a1fe service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:36.745477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6218bca-c440-4be0-a7a4-aa124245a1fe service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:36.745961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6218bca-c440-4be0-a7a4-aa124245a1fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:36.757373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6218bca-c440-4be0-a7a4-aa124245a1fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:36.759307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6218bca-c440-4be0-a7a4-aa124245a1fe service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:36.771003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 673/1360] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:39.111697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826c5df-dafc-4688-93fd-963ae5764dd6 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:39.112228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826c5df-dafc-4688-93fd-963ae5764dd6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:39.112310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826c5df-dafc-4688-93fd-963ae5764dd6 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:39.113840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 688/1361] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:39.119163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea191a69-954e-47c9-a527-877291ddf06e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:39.119697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea191a69-954e-47c9-a527-877291ddf06e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:39.119810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea191a69-954e-47c9-a527-877291ddf06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:39.140252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ea191a69-954e-47c9-a527-877291ddf06e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:39.141528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ea191a69-954e-47c9-a527-877291ddf06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:39.150043 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 674/1362] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:39.194536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826c5df-dafc-4688-93fd-963ae5764dd6 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:39.210235 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826c5df-dafc-4688-93fd-963ae5764dd6 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:39.218641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515ae4b-161f-480c-8970-d158d9de1dac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:39.219366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ae4b-161f-480c-8970-d158d9de1dac service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:39.219720 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ae4b-161f-480c-8970-d158d9de1dac service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:39.220068 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515ae4b-161f-480c-8970-d158d9de1dac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:39.229515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515ae4b-161f-480c-8970-d158d9de1dac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:39.231232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515ae4b-161f-480c-8970-d158d9de1dac service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:39.242802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 689/1363] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:41.938081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfa807-183d-4309-9973-8ce00403e57c None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:41.938609 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfa807-183d-4309-9973-8ce00403e57c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:41.938723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfa807-183d-4309-9973-8ce00403e57c None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:41.940021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 675/1364] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:41.946247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc605a5-41b0-468a-ba47-731781f09baf None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:41.947429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc605a5-41b0-468a-ba47-731781f09baf None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:41.948136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc605a5-41b0-468a-ba47-731781f09baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:41.985701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-fcc605a5-41b0-468a-ba47-731781f09baf None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:41.988061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-fcc605a5-41b0-468a-ba47-731781f09baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:41.997355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 690/1365] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:42.045345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1c75f-38ec-4439-8291-c3275d9e697a None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:42.045635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1c75f-38ec-4439-8291-c3275d9e697a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:42.045913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1c75f-38ec-4439-8291-c3275d9e697a None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:42.047267 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 676/1366] 10.222.0.81 () {54 vars in 961 bytes} [Thu Jun 16 18:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:42.052790 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf68d8f-1e16-44ce-972e-5bed88e5dc9a None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:42.053500 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf68d8f-1e16-44ce-972e-5bed88e5dc9a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:42.053625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf68d8f-1e16-44ce-972e-5bed88e5dc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:42.072333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-fdf68d8f-1e16-44ce-972e-5bed88e5dc9a None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:42.073689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-fdf68d8f-1e16-44ce-972e-5bed88e5dc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:42.083912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 691/1367] 10.222.0.81 () {58 vars in 1058 bytes} [Thu Jun 16 18:48:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:42.091621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1c75f-38ec-4439-8291-c3275d9e697a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:42.105553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1c75f-38ec-4439-8291-c3275d9e697a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:42.110989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e8990-6f24-4c01-b700-b2cccb42c6ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:42.111655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e8990-6f24-4c01-b700-b2cccb42c6ac service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:42.111937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e8990-6f24-4c01-b700-b2cccb42c6ac service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:42.112226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e8990-6f24-4c01-b700-b2cccb42c6ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:42.120182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e8990-6f24-4c01-b700-b2cccb42c6ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:42.121870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e8990-6f24-4c01-b700-b2cccb42c6ac service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:42.131567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 677/1368] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:44.466114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06abeb-ec53-43f2-a185-a526cb230f72 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:44.467155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06abeb-ec53-43f2-a185-a526cb230f72 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:44.467927 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc06abeb-ec53-43f2-a185-a526cb230f72 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:44.475063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28019820-9ee2-4fbd-a828-1d9541ebfc2b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:44.475539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28019820-9ee2-4fbd-a828-1d9541ebfc2b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:44.475621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28019820-9ee2-4fbd-a828-1d9541ebfc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:44.477630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 692/1369] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:44 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:44.493892 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-28019820-9ee2-4fbd-a828-1d9541ebfc2b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:44.495103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-28019820-9ee2-4fbd-a828-1d9541ebfc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:44.503687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 678/1370] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:48:44.548175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06abeb-ec53-43f2-a185-a526cb230f72 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:44.563251 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc06abeb-ec53-43f2-a185-a526cb230f72 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:44.568928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a9e8a-43c8-4fad-af46-8c92de9ac7f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:44.569943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a9e8a-43c8-4fad-af46-8c92de9ac7f8 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:44.570224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a9e8a-43c8-4fad-af46-8c92de9ac7f8 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:44.570517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a9e8a-43c8-4fad-af46-8c92de9ac7f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:44.580698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a9e8a-43c8-4fad-af46-8c92de9ac7f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:44.582376 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a9e8a-43c8-4fad-af46-8c92de9ac7f8 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:44.595552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 693/1371] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:44.633591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45d232-505f-4446-be7f-391d6458d64e None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:44.633591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45d232-505f-4446-be7f-391d6458d64e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:44.633591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45d232-505f-4446-be7f-391d6458d64e None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:44.633983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 679/1372] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:44.639618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664eda6b-90d7-4b4f-880e-79b75627c9ae None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:44.639910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664eda6b-90d7-4b4f-880e-79b75627c9ae None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:44.640189 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664eda6b-90d7-4b4f-880e-79b75627c9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:44.658827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-664eda6b-90d7-4b4f-880e-79b75627c9ae None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:44.660016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-664eda6b-90d7-4b4f-880e-79b75627c9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:44.668025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 694/1373] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:44.974834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add05fde-8647-432b-a282-ffdb26a4e330 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:44.974834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add05fde-8647-432b-a282-ffdb26a4e330 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:44.974834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add05fde-8647-432b-a282-ffdb26a4e330 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:44.984484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 680/1374] 10.222.0.81 () {54 vars in 963 bytes} [Thu Jun 16 18:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:44.985073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846309f5-d030-488c-beeb-b3cc8a255ab6 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:44.985073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846309f5-d030-488c-beeb-b3cc8a255ab6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:44.985073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846309f5-d030-488c-beeb-b3cc8a255ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:44.990224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:48:45.034907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-846309f5-d030-488c-beeb-b3cc8a255ab6 None None] MFA Rules not processed for user `9c82335b7ec74fb3a1508a2fc76335ab`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:45.036108 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-846309f5-d030-488c-beeb-b3cc8a255ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:45.102632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 695/1375] 10.222.0.81 () {58 vars in 1060 bytes} [Thu Jun 16 18:48:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:48:45.111325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add05fde-8647-432b-a282-ffdb26a4e330 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:45.122726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add05fde-8647-432b-a282-ffdb26a4e330 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:45.126291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805f75d-fce2-4ee8-83f0-fc543817196f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:45.126929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805f75d-fce2-4ee8-83f0-fc543817196f service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:45.127210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805f75d-fce2-4ee8-83f0-fc543817196f service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:45.127486 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805f75d-fce2-4ee8-83f0-fc543817196f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:45.137252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805f75d-fce2-4ee8-83f0-fc543817196f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:45.138876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805f75d-fce2-4ee8-83f0-fc543817196f service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:45.149376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 681/1376] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:46.902700 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007243f-b67b-4523-8f7d-e90c7366aa6d None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:46.904355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007243f-b67b-4523-8f7d-e90c7366aa6d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:46.904355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007243f-b67b-4523-8f7d-e90c7366aa6d None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:46.905504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 696/1377] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:46.912179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951c710-835f-4928-95b0-3bac79983448 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:46.913585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951c710-835f-4928-95b0-3bac79983448 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:46.913585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951c710-835f-4928-95b0-3bac79983448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:46.935048 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6951c710-835f-4928-95b0-3bac79983448 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:46.936512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6951c710-835f-4928-95b0-3bac79983448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:46.948008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 682/1378] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:47.018362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007243f-b67b-4523-8f7d-e90c7366aa6d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:47.018362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007243f-b67b-4523-8f7d-e90c7366aa6d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:47.019150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c13085-bece-4d2c-b58e-ea165d5cb221 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:47.019798 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13085-bece-4d2c-b58e-ea165d5cb221 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:47.020085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13085-bece-4d2c-b58e-ea165d5cb221 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:47.020362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13085-bece-4d2c-b58e-ea165d5cb221 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:47.029781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c13085-bece-4d2c-b58e-ea165d5cb221 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:47.031392 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c13085-bece-4d2c-b58e-ea165d5cb221 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:47.042204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 697/1379] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:49.482742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddac3f7-4d91-4860-beeb-221f0707e5df None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:49.483268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddac3f7-4d91-4860-beeb-221f0707e5df None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:49.483400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddac3f7-4d91-4860-beeb-221f0707e5df None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:49.484705 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 683/1380] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:49.490342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08c383-aa32-4829-ab7d-30835fd8c947 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:49.490780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08c383-aa32-4829-ab7d-30835fd8c947 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:49.490908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08c383-aa32-4829-ab7d-30835fd8c947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:49.508833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6d08c383-aa32-4829-ab7d-30835fd8c947 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:49.510048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6d08c383-aa32-4829-ab7d-30835fd8c947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:49.518081 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 698/1381] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:49.562607 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddac3f7-4d91-4860-beeb-221f0707e5df None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:49.578296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddac3f7-4d91-4860-beeb-221f0707e5df None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:49.584746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174939bf-3fda-46c0-86a7-f6dfa233dfdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:49.585441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174939bf-3fda-46c0-86a7-f6dfa233dfdb service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:49.585729 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174939bf-3fda-46c0-86a7-f6dfa233dfdb service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:49.586194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174939bf-3fda-46c0-86a7-f6dfa233dfdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:49.595960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174939bf-3fda-46c0-86a7-f6dfa233dfdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:49.597579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174939bf-3fda-46c0-86a7-f6dfa233dfdb service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:49.607815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 684/1382] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:51.893938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb43efa-8883-43f0-9d2b-da0266e5dddb None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:51.900930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb43efa-8883-43f0-9d2b-da0266e5dddb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:51.900930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb43efa-8883-43f0-9d2b-da0266e5dddb None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:51.900930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 699/1383] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:51.901202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9bcb5-289e-49e5-958c-910fd9a81cae None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:51.902315 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9bcb5-289e-49e5-958c-910fd9a81cae None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:51.902315 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9bcb5-289e-49e5-958c-910fd9a81cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:51.921263 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-acb9bcb5-289e-49e5-958c-910fd9a81cae None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:51.922460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-acb9bcb5-289e-49e5-958c-910fd9a81cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:51.930522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 685/1384] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:51.980435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb43efa-8883-43f0-9d2b-da0266e5dddb None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:51.996958 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb43efa-8883-43f0-9d2b-da0266e5dddb None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:52.002399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27beb197-1c20-488f-821b-d0271ac1ffa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:52.003044 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27beb197-1c20-488f-821b-d0271ac1ffa2 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:52.003324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27beb197-1c20-488f-821b-d0271ac1ffa2 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:52.006182 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27beb197-1c20-488f-821b-d0271ac1ffa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:52.012870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27beb197-1c20-488f-821b-d0271ac1ffa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:52.014605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27beb197-1c20-488f-821b-d0271ac1ffa2 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:52.024809 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 700/1385] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:54.227505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c622042-5bb7-45cf-b3c4-eeae99ffb5b7 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:54.228474 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c622042-5bb7-45cf-b3c4-eeae99ffb5b7 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:54.228835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c622042-5bb7-45cf-b3c4-eeae99ffb5b7 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:54.230423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 686/1386] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:54.236043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a068f-4c9d-4d0e-a5a7-990881c86bd8 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:54.240740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a068f-4c9d-4d0e-a5a7-990881c86bd8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:54.240740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a068f-4c9d-4d0e-a5a7-990881c86bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:54.255979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-cc1a068f-4c9d-4d0e-a5a7-990881c86bd8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:54.257307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-cc1a068f-4c9d-4d0e-a5a7-990881c86bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:54.266173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 701/1387] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:54.311463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c622042-5bb7-45cf-b3c4-eeae99ffb5b7 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:54.329716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c622042-5bb7-45cf-b3c4-eeae99ffb5b7 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:54.336096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8e995-3b95-414b-88f8-24560f239f80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:54.336753 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8e995-3b95-414b-88f8-24560f239f80 service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:54.337041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8e995-3b95-414b-88f8-24560f239f80 service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:54.337370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8e995-3b95-414b-88f8-24560f239f80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:54.346763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8e995-3b95-414b-88f8-24560f239f80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:54.348360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8e995-3b95-414b-88f8-24560f239f80 service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:54.358307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 687/1388] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:56.879554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a9101-2f7c-4dd9-b64b-51ad58e11a1f None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:56.880201 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a9101-2f7c-4dd9-b64b-51ad58e11a1f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:56.880201 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a9101-2f7c-4dd9-b64b-51ad58e11a1f None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:56.881529 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 702/1389] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:56.887485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f664-afe5-4276-a2d2-29d44843fa93 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:56.887991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f664-afe5-4276-a2d2-29d44843fa93 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:56.888128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00f664-afe5-4276-a2d2-29d44843fa93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:56.906102 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-5f00f664-afe5-4276-a2d2-29d44843fa93 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:56.907304 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5f00f664-afe5-4276-a2d2-29d44843fa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:56.915330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 688/1390] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:48:56.959859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a9101-2f7c-4dd9-b64b-51ad58e11a1f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:56.976570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a9101-2f7c-4dd9-b64b-51ad58e11a1f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:56.983675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600174a-a699-4adf-9eb7-199f78a00e3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:56.984511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600174a-a699-4adf-9eb7-199f78a00e3f service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:56.984904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600174a-a699-4adf-9eb7-199f78a00e3f service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:56.985265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600174a-a699-4adf-9eb7-199f78a00e3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:56.996102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600174a-a699-4adf-9eb7-199f78a00e3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:56.998036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600174a-a699-4adf-9eb7-199f78a00e3f service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:57.009803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 703/1391] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:48:59.410411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6050-3218-45d5-bc20-1ede908eb268 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:59.411384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6050-3218-45d5-bc20-1ede908eb268 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:59.411874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb6050-3218-45d5-bc20-1ede908eb268 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:59.413510 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 689/1392] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:48:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:48:59.419044 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fccfdd-e5fb-4666-b762-543c363ebb0d None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:59.419866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fccfdd-e5fb-4666-b762-543c363ebb0d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:59.420371 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fccfdd-e5fb-4666-b762-543c363ebb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:59.438878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-91fccfdd-e5fb-4666-b762-543c363ebb0d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:48:59.440471 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-91fccfdd-e5fb-4666-b762-543c363ebb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:48:59.461266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 704/1393] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:48:59.494688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb6050-3218-45d5-bc20-1ede908eb268 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:48:59.510356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb6050-3218-45d5-bc20-1ede908eb268 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:48:59.515911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfda06e-4812-41b8-8400-b9d8ef451485 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:48:59.516625 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfda06e-4812-41b8-8400-b9d8ef451485 service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:48:59.517001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfda06e-4812-41b8-8400-b9d8ef451485 service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:48:59.517381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfda06e-4812-41b8-8400-b9d8ef451485 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:48:59.526590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfda06e-4812-41b8-8400-b9d8ef451485 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:48:59.528299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfda06e-4812-41b8-8400-b9d8ef451485 service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:48:59.538568 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 690/1394] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:00.945606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf40ba-cf9b-46af-8c80-6095af891615 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:00.946199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf40ba-cf9b-46af-8c80-6095af891615 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:00.946199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf40ba-cf9b-46af-8c80-6095af891615 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:00.948081 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 705/1395] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:49:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:00.954867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4536d6-5c76-488d-8f4d-fe46690c75eb None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:00.955290 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4536d6-5c76-488d-8f4d-fe46690c75eb None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:00.955425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4536d6-5c76-488d-8f4d-fe46690c75eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:00.972786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-9b4536d6-5c76-488d-8f4d-fe46690c75eb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:00.974002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-9b4536d6-5c76-488d-8f4d-fe46690c75eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:00.981836 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 691/1396] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:49:01.037174 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700d600-cc25-4288-94fb-c16b6ab0280d None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:01.037499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700d600-cc25-4288-94fb-c16b6ab0280d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:01.037781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700d600-cc25-4288-94fb-c16b6ab0280d None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:01.039143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 706/1397] 10.222.0.81 () {54 vars in 962 bytes} [Thu Jun 16 18:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:01.044476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39224f14-de32-4c49-8063-2ea26011d483 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:01.044844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39224f14-de32-4c49-8063-2ea26011d483 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:01.045123 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39224f14-de32-4c49-8063-2ea26011d483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:01.093396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-39224f14-de32-4c49-8063-2ea26011d483 None None] MFA Rules not processed for user `d17b7a7128d64b2ba548add02d3ec264`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:01.094585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-39224f14-de32-4c49-8063-2ea26011d483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:01.153970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 692/1398] 10.222.0.81 () {58 vars in 1059 bytes} [Thu Jun 16 18:49:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:01.161531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700d600-cc25-4288-94fb-c16b6ab0280d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:01.172769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700d600-cc25-4288-94fb-c16b6ab0280d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:01.176288 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82179ab0-0850-4731-85f5-3ee8724b8ec3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17b7a7128d64b2ba548add02d3ec264', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:01.176921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82179ab0-0850-4731-85f5-3ee8724b8ec3 service glance] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:01.177208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82179ab0-0850-4731-85f5-3ee8724b8ec3 service glance] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:01.177516 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82179ab0-0850-4731-85f5-3ee8724b8ec3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:01.186903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82179ab0-0850-4731-85f5-3ee8724b8ec3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:01.188350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82179ab0-0850-4731-85f5-3ee8724b8ec3 service glance] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:01.197817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 707/1399] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:01.822497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215f71b-7730-4e9c-8d5f-f28b61f9bf0a None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:01.822810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215f71b-7730-4e9c-8d5f-f28b61f9bf0a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:01.823118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215f71b-7730-4e9c-8d5f-f28b61f9bf0a None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:01.824577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 693/1400] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:01.830073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef38ed-0dc5-4273-9778-2ad5eb4027a9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:01.830366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef38ed-0dc5-4273-9778-2ad5eb4027a9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:01.830643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef38ed-0dc5-4273-9778-2ad5eb4027a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:01.848444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-a0ef38ed-0dc5-4273-9778-2ad5eb4027a9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:01.849690 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a0ef38ed-0dc5-4273-9778-2ad5eb4027a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:01.859709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 708/1401] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:01.902479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215f71b-7730-4e9c-8d5f-f28b61f9bf0a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:01.910349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=93 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:01.916873 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215f71b-7730-4e9c-8d5f-f28b61f9bf0a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:01.922218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a6b83-ff9c-4901-bcf8-6a85c0bc6133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:01.940998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a6b83-ff9c-4901-bcf8-6a85c0bc6133 service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:01.940998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a6b83-ff9c-4901-bcf8-6a85c0bc6133 service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:01.940998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a6b83-ff9c-4901-bcf8-6a85c0bc6133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:01.940998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a6b83-ff9c-4901-bcf8-6a85c0bc6133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:01.940998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3a6b83-ff9c-4901-bcf8-6a85c0bc6133 service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:01.942533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 694/1402] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:02.133350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd994907-5d20-4910-82a6-9f5a8e9d6cc8 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:02.134313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd994907-5d20-4910-82a6-9f5a8e9d6cc8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:02.134444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd994907-5d20-4910-82a6-9f5a8e9d6cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:02.190663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-fd994907-5d20-4910-82a6-9f5a8e9d6cc8 None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:02.194419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-fd994907-5d20-4910-82a6-9f5a8e9d6cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:02.233432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:02.270440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 709/1403] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 18:49:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:02.317390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d1a0-f147-46c6-b688-58cdbb02c664 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:02.317926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d1a0-f147-46c6-b688-58cdbb02c664 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:02.318182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501d1a0-f147-46c6-b688-58cdbb02c664 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:02.319592 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 695/1404] 10.222.0.81 () {54 vars in 942 bytes} [Thu Jun 16 18:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:02.327410 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e39db7-9f45-4d12-81be-69f9d78658c5 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:02.327687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e39db7-9f45-4d12-81be-69f9d78658c5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:02.327969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e39db7-9f45-4d12-81be-69f9d78658c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:02.380475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-a8e39db7-9f45-4d12-81be-69f9d78658c5 None None] MFA Rules not processed for user `b90f65f1f04b416286142c081c92a02d`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:02.381814 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a8e39db7-9f45-4d12-81be-69f9d78658c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:02.451828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 710/1405] 10.222.0.81 () {58 vars in 1039 bytes} [Thu Jun 16 18:49:02 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:02.460887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501d1a0-f147-46c6-b688-58cdbb02c664 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:02.474501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501d1a0-f147-46c6-b688-58cdbb02c664 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:02.479562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f5f25-0ac7-432c-80cc-b8324e693cc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:02.480227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f5f25-0ac7-432c-80cc-b8324e693cc8 service swift] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:02.480517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f5f25-0ac7-432c-80cc-b8324e693cc8 service swift] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:02.480792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f5f25-0ac7-432c-80cc-b8324e693cc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:02.489073 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f5f25-0ac7-432c-80cc-b8324e693cc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:02.490692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f5f25-0ac7-432c-80cc-b8324e693cc8 service swift] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:02.499267 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 696/1406] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 18:49:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:49:04.293150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8b88-ab55-4bfd-b95d-5903a9191070 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:04.293791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8b88-ab55-4bfd-b95d-5903a9191070 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:04.293791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8b88-ab55-4bfd-b95d-5903a9191070 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:04.295083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 711/1407] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:04.301612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3b621-7bd0-4528-bc68-62d3bf773b78 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:04.302119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3b621-7bd0-4528-bc68-62d3bf773b78 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:04.302235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf3b621-7bd0-4528-bc68-62d3bf773b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:04.321533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-fdf3b621-7bd0-4528-bc68-62d3bf773b78 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:04.323041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-fdf3b621-7bd0-4528-bc68-62d3bf773b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:04.331547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 697/1408] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:04.378010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c8b88-ab55-4bfd-b95d-5903a9191070 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:04.394709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c8b88-ab55-4bfd-b95d-5903a9191070 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:04.400308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c837b1-d768-472d-b157-09c7b99c69ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:04.401036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c837b1-d768-472d-b157-09c7b99c69ed service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:04.401353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c837b1-d768-472d-b157-09c7b99c69ed service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:04.401635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c837b1-d768-472d-b157-09c7b99c69ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:04.412447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c837b1-d768-472d-b157-09c7b99c69ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:04.414090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c837b1-d768-472d-b157-09c7b99c69ed service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:04.424211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 712/1409] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:14.316060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=14 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:14.420039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=62 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:26.477311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fcb12-a544-4ee3-9e1b-3bc748466f29 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:26.477901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fcb12-a544-4ee3-9e1b-3bc748466f29 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:26.477901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310fcb12-a544-4ee3-9e1b-3bc748466f29 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:26.479234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 698/1410] 10.222.0.81 () {54 vars in 918 bytes} [Thu Jun 16 18:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:26.484180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d835c1-140b-4337-a1ff-488c5b329457 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:26.484625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d835c1-140b-4337-a1ff-488c5b329457 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:26.484767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d835c1-140b-4337-a1ff-488c5b329457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:26.502206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-b0d835c1-140b-4337-a1ff-488c5b329457 None None] MFA Rules not processed for user `20c0fd976e684356b6143dc33fb7b4af`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:26.503395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b0d835c1-140b-4337-a1ff-488c5b329457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:26.512101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 713/1411] 10.222.0.81 () {58 vars in 1015 bytes} [Thu Jun 16 18:49:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:49:26.526358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccdf1b-0234-4a8a-9981-064d151f372e None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:26.526649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccdf1b-0234-4a8a-9981-064d151f372e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:26.526928 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccdf1b-0234-4a8a-9981-064d151f372e None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:26.528261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 699/1412] 10.222.0.81 () {54 vars in 918 bytes} [Thu Jun 16 18:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:26.532570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a353c01-694f-43e9-94bf-719395b5fe8e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:26.532865 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a353c01-694f-43e9-94bf-719395b5fe8e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:26.533148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a353c01-694f-43e9-94bf-719395b5fe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:26.551858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3a353c01-694f-43e9-94bf-719395b5fe8e None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:26.553042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3a353c01-694f-43e9-94bf-719395b5fe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:26.562001 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 714/1413] 10.222.0.81 () {58 vars in 1015 bytes} [Thu Jun 16 18:49:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:29.176375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a12b83-2c45-4054-908f-fd3fd5c2670d None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:29.176946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a12b83-2c45-4054-908f-fd3fd5c2670d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:29.176946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a12b83-2c45-4054-908f-fd3fd5c2670d None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:29.178349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:49:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:29.183483 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f8dab-bda5-4987-a95e-a40c874799b8 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:29.183939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f8dab-bda5-4987-a95e-a40c874799b8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:29.184043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f8dab-bda5-4987-a95e-a40c874799b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:29.202097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4d3f8dab-bda5-4987-a95e-a40c874799b8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:29.203310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4d3f8dab-bda5-4987-a95e-a40c874799b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:29.211696 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:49:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:29.254160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a12b83-2c45-4054-908f-fd3fd5c2670d None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:29.268207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a12b83-2c45-4054-908f-fd3fd5c2670d None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:29.274086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93137a2b-44ca-4d8f-903f-fbc91d4cd92f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:29.274712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93137a2b-44ca-4d8f-903f-fbc91d4cd92f service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:29.274994 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93137a2b-44ca-4d8f-903f-fbc91d4cd92f service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:29.275269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93137a2b-44ca-4d8f-903f-fbc91d4cd92f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:29.283222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93137a2b-44ca-4d8f-903f-fbc91d4cd92f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:29.284802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93137a2b-44ca-4d8f-903f-fbc91d4cd92f service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:29.294997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 701/1416] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:30.250691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c615a-d5b8-4109-964c-b1c4a132de04 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:30.251540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c615a-d5b8-4109-964c-b1c4a132de04 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:30.251715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c615a-d5b8-4109-964c-b1c4a132de04 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:30.253169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:30.258293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd947fe-c117-4823-b8ee-805d97691898 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:30.258728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd947fe-c117-4823-b8ee-805d97691898 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:30.258859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd947fe-c117-4823-b8ee-805d97691898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:30.276910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-8dd947fe-c117-4823-b8ee-805d97691898 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:30.278132 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8dd947fe-c117-4823-b8ee-805d97691898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:30.286404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:49:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:30.455930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c615a-d5b8-4109-964c-b1c4a132de04 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:30.470438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c615a-d5b8-4109-964c-b1c4a132de04 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:30.475732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023af1cd-8419-4649-98c2-f34d0fd99fae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:30.476366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af1cd-8419-4649-98c2-f34d0fd99fae service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:30.476644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af1cd-8419-4649-98c2-f34d0fd99fae service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:30.476917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023af1cd-8419-4649-98c2-f34d0fd99fae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:30.484999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023af1cd-8419-4649-98c2-f34d0fd99fae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:30.486613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023af1cd-8419-4649-98c2-f34d0fd99fae service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:30.495820 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 717/1419] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:31.420592 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4d932-6c22-4d54-89f2-f8b7d98c7a85 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:31.421031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4d932-6c22-4d54-89f2-f8b7d98c7a85 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:31.421147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4d932-6c22-4d54-89f2-f8b7d98c7a85 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:31.422553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:31.428174 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819feb1-acdb-4202-b663-70a36d0a36c8 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:31.428601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819feb1-acdb-4202-b663-70a36d0a36c8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:31.428740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819feb1-acdb-4202-b663-70a36d0a36c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:31.446792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6819feb1-acdb-4202-b663-70a36d0a36c8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:31.447982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6819feb1-acdb-4202-b663-70a36d0a36c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:31.455875 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:31.606352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4d932-6c22-4d54-89f2-f8b7d98c7a85 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:31.620573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4d932-6c22-4d54-89f2-f8b7d98c7a85 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:31.625899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e518c90-cf7a-418a-97bf-b411137b4d44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:31.631139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518c90-cf7a-418a-97bf-b411137b4d44 service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:31.631139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518c90-cf7a-418a-97bf-b411137b4d44 service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:31.631139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518c90-cf7a-418a-97bf-b411137b4d44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:31.635361 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e518c90-cf7a-418a-97bf-b411137b4d44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:31.636967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e518c90-cf7a-418a-97bf-b411137b4d44 service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:31.646859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 704/1422] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:32.554446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c574e-023a-4e44-8f0e-c98c65b98fc2 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:32.555022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c574e-023a-4e44-8f0e-c98c65b98fc2 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:32.555022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c574e-023a-4e44-8f0e-c98c65b98fc2 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:32.556408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:32.561204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f5d3f-74c5-4377-bb33-9b08fbcfb4a6 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:32.561636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f5d3f-74c5-4377-bb33-9b08fbcfb4a6 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:32.561776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f5d3f-74c5-4377-bb33-9b08fbcfb4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:32.579806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-3c5f5d3f-74c5-4377-bb33-9b08fbcfb4a6 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:32.586662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-3c5f5d3f-74c5-4377-bb33-9b08fbcfb4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:32.589265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:49:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:32.738443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c574e-023a-4e44-8f0e-c98c65b98fc2 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:32.752688 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c574e-023a-4e44-8f0e-c98c65b98fc2 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:32.758794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04d0ae-fc9b-4a7d-bd1e-740117bef23b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:32.759430 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d0ae-fc9b-4a7d-bd1e-740117bef23b service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:32.759714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d0ae-fc9b-4a7d-bd1e-740117bef23b service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:32.759989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04d0ae-fc9b-4a7d-bd1e-740117bef23b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:32.768054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04d0ae-fc9b-4a7d-bd1e-740117bef23b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:32.770224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04d0ae-fc9b-4a7d-bd1e-740117bef23b service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:32.779270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 720/1425] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:33.679446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818860d-c6d8-4045-9130-cc6cba2a895a None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:33.679965 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818860d-c6d8-4045-9130-cc6cba2a895a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:33.679965 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818860d-c6d8-4045-9130-cc6cba2a895a None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:33.681385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:33.686206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7497c61-b882-4086-9bf6-3369d08d4b46 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:33.686632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7497c61-b882-4086-9bf6-3369d08d4b46 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:33.686767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7497c61-b882-4086-9bf6-3369d08d4b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:33.704499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-b7497c61-b882-4086-9bf6-3369d08d4b46 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:33.706321 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b7497c61-b882-4086-9bf6-3369d08d4b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:33.713979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:49:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:33.864357 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818860d-c6d8-4045-9130-cc6cba2a895a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:33.878201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818860d-c6d8-4045-9130-cc6cba2a895a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:33.883438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c144d1-641b-45df-9594-630e9688624b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:33.884062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c144d1-641b-45df-9594-630e9688624b service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:33.884340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c144d1-641b-45df-9594-630e9688624b service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:33.884614 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c144d1-641b-45df-9594-630e9688624b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:33.892482 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c144d1-641b-45df-9594-630e9688624b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:33.894568 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c144d1-641b-45df-9594-630e9688624b service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:33.903644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 707/1428] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:34.824115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca023a-51c4-467a-835c-708acee9cff0 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:34.824639 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca023a-51c4-467a-835c-708acee9cff0 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:34.824738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca023a-51c4-467a-835c-708acee9cff0 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:34.826077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:34.830900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da341500-e687-432d-932c-f9e98f808f40 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:34.831298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da341500-e687-432d-932c-f9e98f808f40 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:34.831457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da341500-e687-432d-932c-f9e98f808f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:34.849428 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-da341500-e687-432d-932c-f9e98f808f40 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:34.850625 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-da341500-e687-432d-932c-f9e98f808f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:34.858299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:35.010496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca023a-51c4-467a-835c-708acee9cff0 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:35.024533 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca023a-51c4-467a-835c-708acee9cff0 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:35.030035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786345a-9958-4733-bc86-720f809a52ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:35.030669 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786345a-9958-4733-bc86-720f809a52ec service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:35.030950 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786345a-9958-4733-bc86-720f809a52ec service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:35.031256 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786345a-9958-4733-bc86-720f809a52ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:35.039981 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786345a-9958-4733-bc86-720f809a52ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:35.041590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786345a-9958-4733-bc86-720f809a52ec service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:35.051298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 723/1431] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:35.946427 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32967b5-0fe0-4212-8eb3-38ffe8310028 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:35.946984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32967b5-0fe0-4212-8eb3-38ffe8310028 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:35.946984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32967b5-0fe0-4212-8eb3-38ffe8310028 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:35.948345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:35.953099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d4068-c345-44f1-bd91-b8ce8311d6ba None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:35.953580 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d4068-c345-44f1-bd91-b8ce8311d6ba None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:35.953692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d4068-c345-44f1-bd91-b8ce8311d6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:35.971763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-af8d4068-c345-44f1-bd91-b8ce8311d6ba None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:35.972946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-af8d4068-c345-44f1-bd91-b8ce8311d6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:35.980635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:36.132239 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32967b5-0fe0-4212-8eb3-38ffe8310028 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:36.146331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32967b5-0fe0-4212-8eb3-38ffe8310028 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:36.151764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a92dd9-976b-4453-8e7b-246330d928df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:36.152390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a92dd9-976b-4453-8e7b-246330d928df service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:36.152681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a92dd9-976b-4453-8e7b-246330d928df service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:36.152956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a92dd9-976b-4453-8e7b-246330d928df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:36.160936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a92dd9-976b-4453-8e7b-246330d928df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:36.162541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a92dd9-976b-4453-8e7b-246330d928df service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:36.171742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 710/1434] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:37.091369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd94cd-a29c-49ce-9391-28e78aee9125 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:37.091893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd94cd-a29c-49ce-9391-28e78aee9125 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:37.092020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bd94cd-a29c-49ce-9391-28e78aee9125 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:37.093299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:37.098667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf0289-4104-439a-a6c5-774f309b9ae3 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:37.099097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf0289-4104-439a-a6c5-774f309b9ae3 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:37.099248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf0289-4104-439a-a6c5-774f309b9ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:37.116693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-81cf0289-4104-439a-a6c5-774f309b9ae3 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:37.118045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-81cf0289-4104-439a-a6c5-774f309b9ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:37.125711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:37.275450 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd94cd-a29c-49ce-9391-28e78aee9125 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:37.289229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bd94cd-a29c-49ce-9391-28e78aee9125 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:37.294569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2ecd0-451c-4e56-bdde-0786b09c1101 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:37.295195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2ecd0-451c-4e56-bdde-0786b09c1101 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:37.295477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2ecd0-451c-4e56-bdde-0786b09c1101 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:37.295743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2ecd0-451c-4e56-bdde-0786b09c1101 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:37.303551 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2ecd0-451c-4e56-bdde-0786b09c1101 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:37.305114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2ecd0-451c-4e56-bdde-0786b09c1101 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:37.314650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 726/1437] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:38.242689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d3cf8-64e2-41df-97cd-d96d1874ba35 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:38.243250 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d3cf8-64e2-41df-97cd-d96d1874ba35 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:38.243250 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d3cf8-64e2-41df-97cd-d96d1874ba35 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:38.244608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:38.249367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198536d5-4b54-4fdd-afe8-e76ec0356890 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:38.249780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198536d5-4b54-4fdd-afe8-e76ec0356890 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:38.249948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198536d5-4b54-4fdd-afe8-e76ec0356890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:38.267905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-198536d5-4b54-4fdd-afe8-e76ec0356890 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:38.269090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-198536d5-4b54-4fdd-afe8-e76ec0356890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:38.276704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:38.446049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d3cf8-64e2-41df-97cd-d96d1874ba35 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:38.465642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d3cf8-64e2-41df-97cd-d96d1874ba35 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:38.471512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a591e492-ca08-4ab5-927b-54734bf2c38f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:38.472158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591e492-ca08-4ab5-927b-54734bf2c38f service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:38.472441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591e492-ca08-4ab5-927b-54734bf2c38f service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:38.472714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a591e492-ca08-4ab5-927b-54734bf2c38f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:38.481465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591e492-ca08-4ab5-927b-54734bf2c38f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:38.484047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a591e492-ca08-4ab5-927b-54734bf2c38f service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:38.498085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 713/1440] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:40.525626 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0596e-62e6-4a38-9b0a-6e1b3993277b None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:40.526212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0596e-62e6-4a38-9b0a-6e1b3993277b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:40.526212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0596e-62e6-4a38-9b0a-6e1b3993277b None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:40.527472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:40.532661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78f365-1022-41ff-8cf5-35b022f368c6 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:40.533112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78f365-1022-41ff-8cf5-35b022f368c6 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:40.533215 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78f365-1022-41ff-8cf5-35b022f368c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:40.551073 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-7e78f365-1022-41ff-8cf5-35b022f368c6 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:40.552247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-7e78f365-1022-41ff-8cf5-35b022f368c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:40.560176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:49:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:40.598705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0596e-62e6-4a38-9b0a-6e1b3993277b None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:40.609408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0596e-62e6-4a38-9b0a-6e1b3993277b None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:40.613496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a24529-79fe-4a07-b96e-18c54ae720c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:40.614128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a24529-79fe-4a07-b96e-18c54ae720c7 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:40.614415 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a24529-79fe-4a07-b96e-18c54ae720c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:40.614696 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a24529-79fe-4a07-b96e-18c54ae720c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:40.623613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a24529-79fe-4a07-b96e-18c54ae720c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:40.625037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a24529-79fe-4a07-b96e-18c54ae720c7 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:40.634876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 729/1443] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:42.971663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc6515-4982-4919-9195-d706b6d68f61 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:42.972247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc6515-4982-4919-9195-d706b6d68f61 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:42.972247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc6515-4982-4919-9195-d706b6d68f61 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:42.973601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:42.979116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87223bb-98d4-42e3-bf85-85623f4eda4f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:42.979523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87223bb-98d4-42e3-bf85-85623f4eda4f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:42.979663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87223bb-98d4-42e3-bf85-85623f4eda4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:42.996797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-a87223bb-98d4-42e3-bf85-85623f4eda4f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:42.998001 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a87223bb-98d4-42e3-bf85-85623f4eda4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:43.005577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:43.044085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc6515-4982-4919-9195-d706b6d68f61 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:43.045779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6ffc6515-4982-4919-9195-d706b6d68f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:43.058671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc6515-4982-4919-9195-d706b6d68f61 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:43.062113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c172e-5b79-4e6d-8382-ff044482d3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:43.062774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c172e-5b79-4e6d-8382-ff044482d3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:43.063020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c172e-5b79-4e6d-8382-ff044482d3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:43.063292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c172e-5b79-4e6d-8382-ff044482d3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:43.072321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c172e-5b79-4e6d-8382-ff044482d3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:43.073763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c172e-5b79-4e6d-8382-ff044482d3d1 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:43.083320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 716/1446] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:45.630987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11928e-e7c5-4732-a6df-8a7e1b065322 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:45.631528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11928e-e7c5-4732-a6df-8a7e1b065322 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:45.631528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11928e-e7c5-4732-a6df-8a7e1b065322 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:45.633002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:45.638214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbc3a6-05fa-4730-bdaa-0113e58091e2 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:45.638634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbc3a6-05fa-4730-bdaa-0113e58091e2 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:45.638784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cbc3a6-05fa-4730-bdaa-0113e58091e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:45.657850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-61cbc3a6-05fa-4730-bdaa-0113e58091e2 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:45.659055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-61cbc3a6-05fa-4730-bdaa-0113e58091e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:45.666870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:45.706991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11928e-e7c5-4732-a6df-8a7e1b065322 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:45.717803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11928e-e7c5-4732-a6df-8a7e1b065322 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:45.722473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a9b86-ec3d-4f3e-a95e-40747aa7d4e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:45.723109 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a9b86-ec3d-4f3e-a95e-40747aa7d4e7 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:45.723394 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a9b86-ec3d-4f3e-a95e-40747aa7d4e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:45.723670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a9b86-ec3d-4f3e-a95e-40747aa7d4e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:45.733784 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a9b86-ec3d-4f3e-a95e-40747aa7d4e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:45.735267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a9b86-ec3d-4f3e-a95e-40747aa7d4e7 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:45.744869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 732/1449] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:49:47.988060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d66f2-8449-4c21-9336-9ae15f7de2e1 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:47.988598 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d66f2-8449-4c21-9336-9ae15f7de2e1 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:47.988722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d66f2-8449-4c21-9336-9ae15f7de2e1 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:47.990040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:47.995172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126763f-3902-4639-a0b8-e41307daf2d9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:47.995583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126763f-3902-4639-a0b8-e41307daf2d9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:47.995727 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126763f-3902-4639-a0b8-e41307daf2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:48.013563 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-8126763f-3902-4639-a0b8-e41307daf2d9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:48.014750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8126763f-3902-4639-a0b8-e41307daf2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:48.022772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:48.061083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d66f2-8449-4c21-9336-9ae15f7de2e1 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:48.072583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d66f2-8449-4c21-9336-9ae15f7de2e1 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:48.076186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a20d7-e35f-4ea6-b796-3f78884c57f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:48.076801 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a20d7-e35f-4ea6-b796-3f78884c57f0 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:48.077086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a20d7-e35f-4ea6-b796-3f78884c57f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:48.077399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a20d7-e35f-4ea6-b796-3f78884c57f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:48.086468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a20d7-e35f-4ea6-b796-3f78884c57f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:48.092121 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a20d7-e35f-4ea6-b796-3f78884c57f0 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:48.097137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 719/1452] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:50.275774 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593c450-2c23-4e64-b7f0-a9d95caa7f39 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:50.276314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593c450-2c23-4e64-b7f0-a9d95caa7f39 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:50.276314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593c450-2c23-4e64-b7f0-a9d95caa7f39 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:50.277737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:50.283461 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0756301-5069-4609-bad7-f218d14652ad None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:50.283840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0756301-5069-4609-bad7-f218d14652ad None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:50.284018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0756301-5069-4609-bad7-f218d14652ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:50.301363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-c0756301-5069-4609-bad7-f218d14652ad None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:50.302610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c0756301-5069-4609-bad7-f218d14652ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:50.310377 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:50.348528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593c450-2c23-4e64-b7f0-a9d95caa7f39 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:50.359019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593c450-2c23-4e64-b7f0-a9d95caa7f39 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:50.362544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1160b-999e-4ec4-863c-e98290d81506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:50.363179 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1160b-999e-4ec4-863c-e98290d81506 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:50.363466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1160b-999e-4ec4-863c-e98290d81506 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:50.363745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1160b-999e-4ec4-863c-e98290d81506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:50.372616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1160b-999e-4ec4-863c-e98290d81506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:50.374111 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1160b-999e-4ec4-863c-e98290d81506 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:50.383384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 735/1455] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:52.555857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f94429-5d00-4a66-9acd-cc350075d7b1 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:52.556421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f94429-5d00-4a66-9acd-cc350075d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:52.556421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f94429-5d00-4a66-9acd-cc350075d7b1 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:52.557800 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:52.563086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaf569-dd35-4561-9881-b2c542a28857 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:52.563518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaf569-dd35-4561-9881-b2c542a28857 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:52.563645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaf569-dd35-4561-9881-b2c542a28857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:52.582273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6adaf569-dd35-4561-9881-b2c542a28857 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:52.583658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6adaf569-dd35-4561-9881-b2c542a28857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:52.591552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:52.629516 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f94429-5d00-4a66-9acd-cc350075d7b1 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:52.640570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f94429-5d00-4a66-9acd-cc350075d7b1 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:52.644050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b72637-e43f-4068-8046-9315f9e40585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:52.644678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b72637-e43f-4068-8046-9315f9e40585 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:52.644983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b72637-e43f-4068-8046-9315f9e40585 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:52.645230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b72637-e43f-4068-8046-9315f9e40585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:52.654114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b72637-e43f-4068-8046-9315f9e40585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:52.656140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b72637-e43f-4068-8046-9315f9e40585 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:52.665445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 722/1458] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:54.819827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875c49d-f8fc-437f-8490-7c50c8c763f9 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:54.820370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875c49d-f8fc-437f-8490-7c50c8c763f9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:54.820370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875c49d-f8fc-437f-8490-7c50c8c763f9 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:54.821843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:54.827597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f1d0f-6201-4474-b465-0bbc5b08e307 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:54.828015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f1d0f-6201-4474-b465-0bbc5b08e307 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:54.828119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f1d0f-6201-4474-b465-0bbc5b08e307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:54.834045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=70 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:54.846096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-118f1d0f-6201-4474-b465-0bbc5b08e307 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:54.847278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-118f1d0f-6201-4474-b465-0bbc5b08e307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:54.855030 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:54.892193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875c49d-f8fc-437f-8490-7c50c8c763f9 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:54.897490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=73 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:49:54.903061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875c49d-f8fc-437f-8490-7c50c8c763f9 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:54.907207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55476b-3b2e-4e45-bed7-ebb73d5c767b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:54.910244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55476b-3b2e-4e45-bed7-ebb73d5c767b service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:54.910244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55476b-3b2e-4e45-bed7-ebb73d5c767b service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:54.910244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55476b-3b2e-4e45-bed7-ebb73d5c767b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:54.917229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55476b-3b2e-4e45-bed7-ebb73d5c767b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:54.918732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e55476b-3b2e-4e45-bed7-ebb73d5c767b service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:54.928624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 738/1461] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:57.085308 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db521d7-e229-4ca6-b4a9-9b6f44e4ef4f None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:57.085884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db521d7-e229-4ca6-b4a9-9b6f44e4ef4f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:57.085884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db521d7-e229-4ca6-b4a9-9b6f44e4ef4f None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:57.087253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:57.092650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2d33-cdba-4026-8159-ba423f0fc5db None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:57.093099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2d33-cdba-4026-8159-ba423f0fc5db None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:57.093206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2d33-cdba-4026-8159-ba423f0fc5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:57.110814 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-7acc2d33-cdba-4026-8159-ba423f0fc5db None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:57.111994 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7acc2d33-cdba-4026-8159-ba423f0fc5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:57.119687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:57.156599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db521d7-e229-4ca6-b4a9-9b6f44e4ef4f None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:57.167642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db521d7-e229-4ca6-b4a9-9b6f44e4ef4f None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:57.171642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d19184-e395-4209-9462-4d71b97b8426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:57.172265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d19184-e395-4209-9462-4d71b97b8426 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:57.172546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d19184-e395-4209-9462-4d71b97b8426 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:57.172819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d19184-e395-4209-9462-4d71b97b8426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:57.181661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d19184-e395-4209-9462-4d71b97b8426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:57.183098 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d19184-e395-4209-9462-4d71b97b8426 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:57.193696 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 725/1464] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:49:59.320472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5e129-8a0f-4243-adc8-41c6a831175a None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:59.321058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5e129-8a0f-4243-adc8-41c6a831175a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:59.321058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5e129-8a0f-4243-adc8-41c6a831175a None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:59.322418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:49:59.328109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d23e3-e76e-4287-b84a-7dd01856dc76 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:59.328553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d23e3-e76e-4287-b84a-7dd01856dc76 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:59.328659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d23e3-e76e-4287-b84a-7dd01856dc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:59.346411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f85d23e3-e76e-4287-b84a-7dd01856dc76 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:49:59.347603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f85d23e3-e76e-4287-b84a-7dd01856dc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:49:59.355226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:49:59.391238 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5e129-8a0f-4243-adc8-41c6a831175a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:49:59.401773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5e129-8a0f-4243-adc8-41c6a831175a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:49:59.405907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c9f3f-c7a2-4046-83de-41958189e357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:49:59.406543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c9f3f-c7a2-4046-83de-41958189e357 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:49:59.406830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c9f3f-c7a2-4046-83de-41958189e357 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:49:59.407131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c9f3f-c7a2-4046-83de-41958189e357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:49:59.415976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c9f3f-c7a2-4046-83de-41958189e357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:49:59.417437 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c9f3f-c7a2-4046-83de-41958189e357 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:49:59.426819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 741/1467] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:01.586753 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed76f-97a5-4fa5-83b7-7f2f0a426acb None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:01.589304 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed76f-97a5-4fa5-83b7-7f2f0a426acb None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:01.589304 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ed76f-97a5-4fa5-83b7-7f2f0a426acb None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:01.589304 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:01.593941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9c7f6-f559-453c-bd5d-1650c49cd4e9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:01.594359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9c7f6-f559-453c-bd5d-1650c49cd4e9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:01.594513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec9c7f6-f559-453c-bd5d-1650c49cd4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:01.612137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-fec9c7f6-f559-453c-bd5d-1650c49cd4e9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:01.613359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-fec9c7f6-f559-453c-bd5d-1650c49cd4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:01.622448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:01.657403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ed76f-97a5-4fa5-83b7-7f2f0a426acb None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:01.668420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ed76f-97a5-4fa5-83b7-7f2f0a426acb None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:01.671910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43fe8bc-e8e9-48bc-9ef0-ae79721b4494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:01.672538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fe8bc-e8e9-48bc-9ef0-ae79721b4494 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:01.672823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fe8bc-e8e9-48bc-9ef0-ae79721b4494 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:01.673098 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43fe8bc-e8e9-48bc-9ef0-ae79721b4494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:01.682203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43fe8bc-e8e9-48bc-9ef0-ae79721b4494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:01.684261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43fe8bc-e8e9-48bc-9ef0-ae79721b4494 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:01.693907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 728/1470] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:11.607291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=17 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:11.689632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=17 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:21.729350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a17ae-d122-4758-abdb-203481274e8e None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:21.729917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a17ae-d122-4758-abdb-203481274e8e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:21.729917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a17ae-d122-4758-abdb-203481274e8e None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:21.731184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 743/1471] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:21.736477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc7858-c8ce-4a14-90b2-230ef6490eed None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:21.736889 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc7858-c8ce-4a14-90b2-230ef6490eed None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:21.737033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc7858-c8ce-4a14-90b2-230ef6490eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:21.754236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-69bc7858-c8ce-4a14-90b2-230ef6490eed None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:21.755429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-69bc7858-c8ce-4a14-90b2-230ef6490eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:21.764213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 729/1472] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:21.811667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb8c5b-50b1-40f6-9ea8-291da01f443e None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:21.811953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb8c5b-50b1-40f6-9ea8-291da01f443e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:21.812229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb8c5b-50b1-40f6-9ea8-291da01f443e None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:21.813608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 744/1473] 10.222.0.81 () {54 vars in 962 bytes} [Thu Jun 16 18:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:21.818210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51294b-cd29-412a-8ec9-220fb6c5c29e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:21.818501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51294b-cd29-412a-8ec9-220fb6c5c29e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:21.818782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51294b-cd29-412a-8ec9-220fb6c5c29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:21.837310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2c51294b-cd29-412a-8ec9-220fb6c5c29e None None] MFA Rules not processed for user `d17b7a7128d64b2ba548add02d3ec264`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:21.838491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2c51294b-cd29-412a-8ec9-220fb6c5c29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:21.844775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 730/1474] 10.222.0.81 () {58 vars in 1059 bytes} [Thu Jun 16 18:50:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:21.851896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb8c5b-50b1-40f6-9ea8-291da01f443e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:21.862117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb8c5b-50b1-40f6-9ea8-291da01f443e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:21.865524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c0ecc-cb2d-447e-ac69-a6fefa4a9d50 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17b7a7128d64b2ba548add02d3ec264', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:21.866160 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0ecc-cb2d-447e-ac69-a6fefa4a9d50 service glance] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:21.866418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0ecc-cb2d-447e-ac69-a6fefa4a9d50 service glance] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:21.866761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0ecc-cb2d-447e-ac69-a6fefa4a9d50 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:21.875262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c0ecc-cb2d-447e-ac69-a6fefa4a9d50 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:21.876675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c0ecc-cb2d-447e-ac69-a6fefa4a9d50 service glance] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:21.886481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 745/1475] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:24.113038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72f45d-3f48-44d4-96dd-570c78cde362 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:24.113650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72f45d-3f48-44d4-96dd-570c78cde362 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:24.113650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72f45d-3f48-44d4-96dd-570c78cde362 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:24.115012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 731/1476] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:24.120659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67236ecb-ef34-477f-89ed-9378d0d20b06 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:24.121084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67236ecb-ef34-477f-89ed-9378d0d20b06 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:24.121209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67236ecb-ef34-477f-89ed-9378d0d20b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:24.138369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-67236ecb-ef34-477f-89ed-9378d0d20b06 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:24.139552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-67236ecb-ef34-477f-89ed-9378d0d20b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:24.147573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 746/1477] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:24.170912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72f45d-3f48-44d4-96dd-570c78cde362 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:24.183666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72f45d-3f48-44d4-96dd-570c78cde362 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:24.188288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13049c-3beb-4e55-b0f1-b2aff43db108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:24.188918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13049c-3beb-4e55-b0f1-b2aff43db108 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:24.189201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13049c-3beb-4e55-b0f1-b2aff43db108 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:24.189501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13049c-3beb-4e55-b0f1-b2aff43db108 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:24.190205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13049c-3beb-4e55-b0f1-b2aff43db108 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:24.191942 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: WARNING py.warnings [None req-6c13049c-3beb-4e55-b0f1-b2aff43db108 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:50:24.191942 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: warnings.warn(msg) Jun 16 18:50:24.191942 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: Jun 16 18:50:24.193077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13049c-3beb-4e55-b0f1-b2aff43db108 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:24.211618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 732/1478] 10.222.0.81 () {56 vars in 1108 bytes} [Thu Jun 16 18:50:24 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:50:26.356684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4afb62-18c5-4c6c-a191-d12f9d6a40e7 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:26.357263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4afb62-18c5-4c6c-a191-d12f9d6a40e7 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:26.357263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4afb62-18c5-4c6c-a191-d12f9d6a40e7 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:26.358643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 747/1479] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:26.363789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfc6b6-0b0a-4ad6-96bd-005fa2180872 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:26.364221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfc6b6-0b0a-4ad6-96bd-005fa2180872 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:26.364361 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfc6b6-0b0a-4ad6-96bd-005fa2180872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:26.382355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f3bfc6b6-0b0a-4ad6-96bd-005fa2180872 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:26.383562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f3bfc6b6-0b0a-4ad6-96bd-005fa2180872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:26.391366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 733/1480] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:26.433789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4afb62-18c5-4c6c-a191-d12f9d6a40e7 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:26.448482 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4afb62-18c5-4c6c-a191-d12f9d6a40e7 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:26.453946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca446ce-1c44-48c3-9a70-05df50fbef8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:26.454577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca446ce-1c44-48c3-9a70-05df50fbef8e service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:26.454866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca446ce-1c44-48c3-9a70-05df50fbef8e service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:26.455145 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca446ce-1c44-48c3-9a70-05df50fbef8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:26.464227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca446ce-1c44-48c3-9a70-05df50fbef8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:26.465843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca446ce-1c44-48c3-9a70-05df50fbef8e service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:26.475625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 748/1481] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:28.768802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e631b-ad29-4972-9b8f-526aae390e27 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:28.769339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e631b-ad29-4972-9b8f-526aae390e27 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:28.769339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e631b-ad29-4972-9b8f-526aae390e27 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:28.770688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 734/1482] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:28.776335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666fb18-8253-412c-8f0e-94814a66289c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:28.776735 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666fb18-8253-412c-8f0e-94814a66289c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:28.776948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666fb18-8253-412c-8f0e-94814a66289c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:28.795017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-a666fb18-8253-412c-8f0e-94814a66289c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:28.796248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a666fb18-8253-412c-8f0e-94814a66289c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:28.803935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 749/1483] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:28.840813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e631b-ad29-4972-9b8f-526aae390e27 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:28.852070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e631b-ad29-4972-9b8f-526aae390e27 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:28.855560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586ea39-43ac-4009-a0c9-06708717b4fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:28.856194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586ea39-43ac-4009-a0c9-06708717b4fe service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:28.856481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586ea39-43ac-4009-a0c9-06708717b4fe service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:28.856759 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586ea39-43ac-4009-a0c9-06708717b4fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:28.866198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586ea39-43ac-4009-a0c9-06708717b4fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:28.867641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586ea39-43ac-4009-a0c9-06708717b4fe service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:28.876879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 735/1484] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:31.012337 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb07cc9-4187-4443-bc0c-a20cb4850201 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:31.012884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb07cc9-4187-4443-bc0c-a20cb4850201 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:31.012884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb07cc9-4187-4443-bc0c-a20cb4850201 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:31.014268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 750/1485] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:31.019362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46393-a1a4-4a29-bd40-29f889e98059 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:31.019790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46393-a1a4-4a29-bd40-29f889e98059 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:31.019933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46393-a1a4-4a29-bd40-29f889e98059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:31.037660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-8fa46393-a1a4-4a29-bd40-29f889e98059 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:31.038890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8fa46393-a1a4-4a29-bd40-29f889e98059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:31.046644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 736/1486] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:31.084303 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb07cc9-4187-4443-bc0c-a20cb4850201 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:31.095026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb07cc9-4187-4443-bc0c-a20cb4850201 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:31.099152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14accbd9-2bf1-4313-96c1-b67c627ebecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:31.099775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14accbd9-2bf1-4313-96c1-b67c627ebecc service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:31.100055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14accbd9-2bf1-4313-96c1-b67c627ebecc service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:31.100326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14accbd9-2bf1-4313-96c1-b67c627ebecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:31.108945 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14accbd9-2bf1-4313-96c1-b67c627ebecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:31.110394 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14accbd9-2bf1-4313-96c1-b67c627ebecc service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:31.119546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 751/1487] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:33.349257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da4ae3-7d3e-49a0-966e-7a6010f8b16e None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:33.349815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da4ae3-7d3e-49a0-966e-7a6010f8b16e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:33.349815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da4ae3-7d3e-49a0-966e-7a6010f8b16e None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:33.351200 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 737/1488] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:50:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:33.356349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7eac0-b121-42da-8c17-b42bdaefbbc1 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:33.356736 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7eac0-b121-42da-8c17-b42bdaefbbc1 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:33.356940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7eac0-b121-42da-8c17-b42bdaefbbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:33.374329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-43e7eac0-b121-42da-8c17-b42bdaefbbc1 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:33.375521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-43e7eac0-b121-42da-8c17-b42bdaefbbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:33.383324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 752/1489] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:33.423247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da4ae3-7d3e-49a0-966e-7a6010f8b16e None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:33.433930 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da4ae3-7d3e-49a0-966e-7a6010f8b16e None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:33.437384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d934aa-718d-4056-a4e0-c2713c527b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:33.438012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d934aa-718d-4056-a4e0-c2713c527b18 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:33.438300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d934aa-718d-4056-a4e0-c2713c527b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:33.438579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d934aa-718d-4056-a4e0-c2713c527b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:33.447385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d934aa-718d-4056-a4e0-c2713c527b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:33.448813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d934aa-718d-4056-a4e0-c2713c527b18 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:33.459720 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 738/1490] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:36.030773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf408d-90cf-49a6-ac77-45991a028a78 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:36.031305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf408d-90cf-49a6-ac77-45991a028a78 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:36.031305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf408d-90cf-49a6-ac77-45991a028a78 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:36.032599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 753/1491] 10.222.0.81 () {54 vars in 926 bytes} [Thu Jun 16 18:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:50:36.038261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521a084-3db1-45ac-ba48-178ddd31154e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:36.038656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521a084-3db1-45ac-ba48-178ddd31154e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:36.038820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521a084-3db1-45ac-ba48-178ddd31154e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:36.056579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-8521a084-3db1-45ac-ba48-178ddd31154e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:50:36.058055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8521a084-3db1-45ac-ba48-178ddd31154e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:50:36.065656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 739/1492] 10.222.0.81 () {58 vars in 1023 bytes} [Thu Jun 16 18:50:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:50:36.103450 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf408d-90cf-49a6-ac77-45991a028a78 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:50:36.114659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf408d-90cf-49a6-ac77-45991a028a78 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:50:36.118097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c027f-8540-4920-8ce0-d94fa78e0a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:50:36.118731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c027f-8540-4920-8ce0-d94fa78e0a5d service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:50:36.119015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c027f-8540-4920-8ce0-d94fa78e0a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:50:36.119288 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c027f-8540-4920-8ce0-d94fa78e0a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:50:36.127877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c027f-8540-4920-8ce0-d94fa78e0a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:50:36.129360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c027f-8540-4920-8ce0-d94fa78e0a5d service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:50:36.138660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 754/1493] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:50:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:50:46.051629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=33 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:50:46.134491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=33 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:52:10.066927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ef94-1c6b-44a6-b3cf-5e438267fcdd None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.067493 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ef94-1c6b-44a6-b3cf-5e438267fcdd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.067493 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ef94-1c6b-44a6-b3cf-5e438267fcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.085620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f554ef94-1c6b-44a6-b3cf-5e438267fcdd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:10.086805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f554ef94-1c6b-44a6-b3cf-5e438267fcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:10.094459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 740/1494] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:10.103283 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782ae8-6d9a-4f02-8273-e886104d2d5e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.103764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782ae8-6d9a-4f02-8273-e886104d2d5e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.103858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10782ae8-6d9a-4f02-8273-e886104d2d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.121567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-10782ae8-6d9a-4f02-8273-e886104d2d5e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:10.123451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-10782ae8-6d9a-4f02-8273-e886104d2d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:10.131590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 755/1495] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:10.136304 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554ef94-1c6b-44a6-b3cf-5e438267fcdd None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:10.150831 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554ef94-1c6b-44a6-b3cf-5e438267fcdd None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:10.151267 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a098865-db6b-4911-b2de-8e2bf4ef4f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:10.151919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a098865-db6b-4911-b2de-8e2bf4ef4f5a admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.152206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a098865-db6b-4911-b2de-8e2bf4ef4f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.152483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a098865-db6b-4911-b2de-8e2bf4ef4f5a admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.153218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a098865-db6b-4911-b2de-8e2bf4ef4f5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:52:10.153563 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a098865-db6b-4911-b2de-8e2bf4ef4f5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:10.154578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: WARNING py.warnings [None req-6a098865-db6b-4911-b2de-8e2bf4ef4f5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:52:10.154578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: warnings.warn(msg) Jun 16 18:52:10.154578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: Jun 16 18:52:10.155454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a098865-db6b-4911-b2de-8e2bf4ef4f5a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:10.167176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 741/1496] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:52:10.175328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782ae8-6d9a-4f02-8273-e886104d2d5e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:10.185047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10782ae8-6d9a-4f02-8273-e886104d2d5e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:10.189658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aa6ab-3bc7-4500-a98e-1e766f2b602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:10.190304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aa6ab-3bc7-4500-a98e-1e766f2b602c admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.190589 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aa6ab-3bc7-4500-a98e-1e766f2b602c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.190864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aa6ab-3bc7-4500-a98e-1e766f2b602c admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.191543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4aa6ab-3bc7-4500-a98e-1e766f2b602c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:10.193351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: WARNING py.warnings [None req-3d4aa6ab-3bc7-4500-a98e-1e766f2b602c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:52:10.193351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: warnings.warn(msg) Jun 16 18:52:10.193351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: Jun 16 18:52:10.194586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4aa6ab-3bc7-4500-a98e-1e766f2b602c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:10.294109 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 756/1497] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:52:10 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:52:10.298288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a098865-db6b-4911-b2de-8e2bf4ef4f5a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:10.308154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a098865-db6b-4911-b2de-8e2bf4ef4f5a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:10.362498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b2f9a-5442-4b4e-a7d9-8cf26c32d88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:10.363090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b2f9a-5442-4b4e-a7d9-8cf26c32d88d admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.363373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b2f9a-5442-4b4e-a7d9-8cf26c32d88d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.363647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b2f9a-5442-4b4e-a7d9-8cf26c32d88d admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.364319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b2f9a-5442-4b4e-a7d9-8cf26c32d88d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:10.365376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: WARNING py.warnings [None req-809b2f9a-5442-4b4e-a7d9-8cf26c32d88d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:52:10.365376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: warnings.warn(msg) Jun 16 18:52:10.365376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: Jun 16 18:52:10.366578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b2f9a-5442-4b4e-a7d9-8cf26c32d88d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:10.448374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 742/1498] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:52:10 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:52:10.452504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aa6ab-3bc7-4500-a98e-1e766f2b602c admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:10.462693 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aa6ab-3bc7-4500-a98e-1e766f2b602c admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:10.467264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc721682-755f-4d21-9903-3710c586b250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:10.467928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc721682-755f-4d21-9903-3710c586b250 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.468240 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc721682-755f-4d21-9903-3710c586b250 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.468516 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc721682-755f-4d21-9903-3710c586b250 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.469230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc721682-755f-4d21-9903-3710c586b250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:10.470265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: WARNING py.warnings [None req-cc721682-755f-4d21-9903-3710c586b250 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:52:10.470265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: warnings.warn(msg) Jun 16 18:52:10.470265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: Jun 16 18:52:10.471136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc721682-755f-4d21-9903-3710c586b250 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:10.484305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 757/1499] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:52:10.488447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b2f9a-5442-4b4e-a7d9-8cf26c32d88d admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:10.498929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b2f9a-5442-4b4e-a7d9-8cf26c32d88d admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:10.503739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd2d8e-52c6-4dd3-b5b1-b1924e7ff425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:10.504384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd2d8e-52c6-4dd3-b5b1-b1924e7ff425 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.504666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd2d8e-52c6-4dd3-b5b1-b1924e7ff425 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.504939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd2d8e-52c6-4dd3-b5b1-b1924e7ff425 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.505707 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd2d8e-52c6-4dd3-b5b1-b1924e7ff425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:10.506711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: WARNING py.warnings [None req-dfdd2d8e-52c6-4dd3-b5b1-b1924e7ff425 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:52:10.506711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: warnings.warn(msg) Jun 16 18:52:10.506711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: Jun 16 18:52:10.507591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd2d8e-52c6-4dd3-b5b1-b1924e7ff425 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:10.521840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 743/1500] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:52:10.525866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc721682-755f-4d21-9903-3710c586b250 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:10.535465 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc721682-755f-4d21-9903-3710c586b250 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:10.539973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2420b0-d6f5-4b5b-860e-2f1e1e77e0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:10.540867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2420b0-d6f5-4b5b-860e-2f1e1e77e0f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.541154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2420b0-d6f5-4b5b-860e-2f1e1e77e0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.541471 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2420b0-d6f5-4b5b-860e-2f1e1e77e0f6 admin admin] PATH_INFO: `/v3/projects/24c4b96228e948a5b91b1f80b34e21cf/users/1fec0ef5268a4fd59e84e06d577a33d9/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.556214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2420b0-d6f5-4b5b-860e-2f1e1e77e0f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c4b96228e948a5b91b1f80b34e21cf, user_id=1fec0ef5268a4fd59e84e06d577a33d9, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:10.557271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: WARNING py.warnings [None req-ba2420b0-d6f5-4b5b-860e-2f1e1e77e0f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:52:10.557271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: warnings.warn(msg) Jun 16 18:52:10.557271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: Jun 16 18:52:10.564452 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2420b0-d6f5-4b5b-860e-2f1e1e77e0f6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:10.641996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 758/1501] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:52:10 2022] PUT /identity/v3/projects/24c4b96228e948a5b91b1f80b34e21cf/users/1fec0ef5268a4fd59e84e06d577a33d9/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:52:10.653088 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78e39d-e9df-4b80-b679-8dba31eebfef None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.653404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78e39d-e9df-4b80-b679-8dba31eebfef None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.653680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78e39d-e9df-4b80-b679-8dba31eebfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.679172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-5a78e39d-e9df-4b80-b679-8dba31eebfef None None] MFA Rules not processed for user `1fec0ef5268a4fd59e84e06d577a33d9`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:10.680411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5a78e39d-e9df-4b80-b679-8dba31eebfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:10.741748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 744/1502] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:52:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:10.755619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdbb44-3152-458e-b7c2-87f4d11446a5 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.755913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdbb44-3152-458e-b7c2-87f4d11446a5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.756194 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdbb44-3152-458e-b7c2-87f4d11446a5 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.757621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 759/1503] 10.222.0.81 () {54 vars in 963 bytes} [Thu Jun 16 18:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:10.762356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd68c0-a888-4685-aadd-ddb849c9ba09 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.762659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd68c0-a888-4685-aadd-ddb849c9ba09 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.762943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd68c0-a888-4685-aadd-ddb849c9ba09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.780545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-7ccd68c0-a888-4685-aadd-ddb849c9ba09 None None] MFA Rules not processed for user `9c82335b7ec74fb3a1508a2fc76335ab`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:10.781771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-7ccd68c0-a888-4685-aadd-ddb849c9ba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:10.826237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:52:10.833060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 745/1504] 10.222.0.81 () {58 vars in 1060 bytes} [Thu Jun 16 18:52:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:10.840795 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdbb44-3152-458e-b7c2-87f4d11446a5 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:10.852124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdbb44-3152-458e-b7c2-87f4d11446a5 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:10.855734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e787c5-f584-4de2-85f5-1e2d8dd6f354 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:10.856416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e787c5-f584-4de2-85f5-1e2d8dd6f354 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.856709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e787c5-f584-4de2-85f5-1e2d8dd6f354 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.856991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e787c5-f584-4de2-85f5-1e2d8dd6f354 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.866002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e787c5-f584-4de2-85f5-1e2d8dd6f354 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:10.868896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e787c5-f584-4de2-85f5-1e2d8dd6f354 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:10.876246 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 760/1505] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:10.918433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107ad81-921e-48b2-af89-5fa94635cd12 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.918726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107ad81-921e-48b2-af89-5fa94635cd12 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.919005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107ad81-921e-48b2-af89-5fa94635cd12 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.920854 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 746/1506] 10.222.0.81 () {52 vars in 829 bytes} [Thu Jun 16 18:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:52:10.924471 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e787c5-f584-4de2-85f5-1e2d8dd6f354 service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:10.985661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e787c5-f584-4de2-85f5-1e2d8dd6f354 service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:10.990196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:10.991066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:10.991352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:10.991630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] PATH_INFO: `/v3/users/1fec0ef5268a4fd59e84e06d577a33d9` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:10.992799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fec0ef5268a4fd59e84e06d577a33d9)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:10.993841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: WARNING py.warnings [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:52:10.993841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: warnings.warn(msg) Jun 16 18:52:10.993841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: Jun 16 18:52:10.995038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:11.197599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fec0ef5268a4fd59e84e06d577a33d9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:52:11.198246 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fec0ef5268a4fd59e84e06d577a33d9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:52:11.211116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fec0ef5268a4fd59e84e06d577a33d9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:52:11.211881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fec0ef5268a4fd59e84e06d577a33d9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:52:11.230631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fec0ef5268a4fd59e84e06d577a33d9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:52:11.234538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:52:11.330258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 761/1507] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:52:10 2022] DELETE /identity/v3/users/1fec0ef5268a4fd59e84e06d577a33d9 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:52:11.334615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0ba97-72cd-4720-a4bf-589007e892e2 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:11.335167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0ba97-72cd-4720-a4bf-589007e892e2 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:11.335338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0ba97-72cd-4720-a4bf-589007e892e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:11.353171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-69a0ba97-72cd-4720-a4bf-589007e892e2 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:52:11.354399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-69a0ba97-72cd-4720-a4bf-589007e892e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:11.411939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 747/1508] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:52:11.421461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:11.422911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:11.480869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec6b39-b2e9-4380-9093-7c6a803263fd admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:11.484434 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff028dcd-d90e-4f88-a9e3-97cb3de775b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:11.485080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff028dcd-d90e-4f88-a9e3-97cb3de775b0 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:11.485401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff028dcd-d90e-4f88-a9e3-97cb3de775b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:11.485682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff028dcd-d90e-4f88-a9e3-97cb3de775b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:11.495118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff028dcd-d90e-4f88-a9e3-97cb3de775b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:11.496553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff028dcd-d90e-4f88-a9e3-97cb3de775b0 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:11.506594 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 762/1509] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:52:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:52:11.967237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0ba97-72cd-4720-a4bf-589007e892e2 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:52:11.968815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-69a0ba97-72cd-4720-a4bf-589007e892e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:52:11.981086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0ba97-72cd-4720-a4bf-589007e892e2 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:52:11.986472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:52:11.987431 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:52:11.987700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:52:11.987975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] PATH_INFO: `/v3/projects/24c4b96228e948a5b91b1f80b34e21cf` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:52:11.989212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c4b96228e948a5b91b1f80b34e21cf)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:52:11.990271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: WARNING py.warnings [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:52:11.990271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: warnings.warn(msg) Jun 16 18:52:11.990271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: Jun 16 18:52:11.991500 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:52:12.131568 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c4b96228e948a5b91b1f80b34e21cf', 'request_id': 'req-728811b7-2935-4e58-affb-ffdf51fcee79'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:52:12.194569 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c4b96228e948a5b91b1f80b34e21cf', 'request_id': 'req-728811b7-2935-4e58-affb-ffdf51fcee79'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:52:12.201479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c4b96228e948a5b91b1f80b34e21cf', 'request_id': 'req-728811b7-2935-4e58-affb-ffdf51fcee79'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:52:12.220325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] The token cache is being invalidate because project 24c4b96228e948a5b91b1f80b34e21cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:52:12.220614 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:52:12.221275 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-728811b7-2935-4e58-affb-ffdf51fcee79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:52:12.222935 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 748/1510] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:52:11 2022] DELETE /identity/v3/projects/24c4b96228e948a5b91b1f80b34e21cf => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:52:21.502272 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:52:22.217370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:53:51.497148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746a154-3a7f-4aa6-b301-5e743031e231 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:51.497763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746a154-3a7f-4aa6-b301-5e743031e231 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:51.497763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746a154-3a7f-4aa6-b301-5e743031e231 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:51.499114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:51.503928 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26beba-695a-4efc-960a-7f3d83b688d2 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:51.504371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26beba-695a-4efc-960a-7f3d83b688d2 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:51.504480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26beba-695a-4efc-960a-7f3d83b688d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:51.522405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6c26beba-695a-4efc-960a-7f3d83b688d2 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:51.523591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6c26beba-695a-4efc-960a-7f3d83b688d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:51.580101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:51.746824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746a154-3a7f-4aa6-b301-5e743031e231 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:51.754853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6746a154-3a7f-4aa6-b301-5e743031e231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:51.822868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746a154-3a7f-4aa6-b301-5e743031e231 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:51.828660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17304d04-4f0f-41d6-8377-7b072cc70441 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:51.829328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17304d04-4f0f-41d6-8377-7b072cc70441 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:51.829612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17304d04-4f0f-41d6-8377-7b072cc70441 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:51.829885 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17304d04-4f0f-41d6-8377-7b072cc70441 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:51.837976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17304d04-4f0f-41d6-8377-7b072cc70441 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:51.840167 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17304d04-4f0f-41d6-8377-7b072cc70441 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:51.849215 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 764/1513] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:52.727380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886baac-5068-444c-9eb2-2a77445bcda2 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:52.727917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886baac-5068-444c-9eb2-2a77445bcda2 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:52.727917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886baac-5068-444c-9eb2-2a77445bcda2 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:52.729220 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:52.735447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5572-6af1-4ac0-8909-a58efb0d2720 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:52.735900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5572-6af1-4ac0-8909-a58efb0d2720 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:52.736007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5572-6af1-4ac0-8909-a58efb0d2720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:52.872407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-eaaf5572-6af1-4ac0-8909-a58efb0d2720 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:52.873682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-eaaf5572-6af1-4ac0-8909-a58efb0d2720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:52.881518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:53.042189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886baac-5068-444c-9eb2-2a77445bcda2 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:53.057042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886baac-5068-444c-9eb2-2a77445bcda2 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:53.063177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a1f9f-d388-46a1-8f37-fe674895aff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:53.064205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a1f9f-d388-46a1-8f37-fe674895aff2 service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:53.064492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a1f9f-d388-46a1-8f37-fe674895aff2 service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:53.064936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a1f9f-d388-46a1-8f37-fe674895aff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:53.076118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a1f9f-d388-46a1-8f37-fe674895aff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:53.077963 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a1f9f-d388-46a1-8f37-fe674895aff2 service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:53.088708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 751/1516] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:53:53.375973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28014b-312a-4c51-9ee1-3d69e2052367 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:53.376265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28014b-312a-4c51-9ee1-3d69e2052367 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:53.376567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28014b-312a-4c51-9ee1-3d69e2052367 None None] PATH_INFO: `/v3` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:53.377909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 766/1517] 10.222.0.66 () {54 vars in 925 bytes} [Thu Jun 16 18:53:53 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:53:53.384179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1414b48-535a-4671-9378-e1da88134593 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:53.384454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1414b48-535a-4671-9378-e1da88134593 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:53.384738 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1414b48-535a-4671-9378-e1da88134593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:53.406545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-c1414b48-535a-4671-9378-e1da88134593 None None] MFA Rules not processed for user `20c0fd976e684356b6143dc33fb7b4af`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:53.407803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c1414b48-535a-4671-9378-e1da88134593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:53.480880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 752/1518] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Jun 16 18:53:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:57.388340 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c58db-56f8-4ae6-8b23-a29fdafea6c6 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:57.393369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c58db-56f8-4ae6-8b23-a29fdafea6c6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:57.393369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c58db-56f8-4ae6-8b23-a29fdafea6c6 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:57.393369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 767/1519] 10.222.0.81 () {54 vars in 960 bytes} [Thu Jun 16 18:53:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:53:57.395690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2627e-5d11-4c2c-aaa5-60fc1da6f0f7 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:57.396105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2627e-5d11-4c2c-aaa5-60fc1da6f0f7 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:57.396248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2627e-5d11-4c2c-aaa5-60fc1da6f0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:57.413743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b5a2627e-5d11-4c2c-aaa5-60fc1da6f0f7 None None] MFA Rules not processed for user `20c0fd976e684356b6143dc33fb7b4af`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:53:57.414948 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b5a2627e-5d11-4c2c-aaa5-60fc1da6f0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:53:57.423364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 753/1520] 10.222.0.81 () {58 vars in 1057 bytes} [Thu Jun 16 18:53:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:53:57.430697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c58db-56f8-4ae6-8b23-a29fdafea6c6 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:53:57.444466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c58db-56f8-4ae6-8b23-a29fdafea6c6 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:53:57.449364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e88b4b-005e-400b-974e-1ac63415460f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:53:57.449994 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e88b4b-005e-400b-974e-1ac63415460f service placement] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:53:57.450846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e88b4b-005e-400b-974e-1ac63415460f service placement] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:53:57.451120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e88b4b-005e-400b-974e-1ac63415460f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:53:57.459574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e88b4b-005e-400b-974e-1ac63415460f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:53:57.461131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e88b4b-005e-400b-974e-1ac63415460f service placement] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:53:57.470605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 768/1521] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:53:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:54:04.344235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b15c5-dad2-46f9-a69d-7fc76e3849c0 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:04.344759 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b15c5-dad2-46f9-a69d-7fc76e3849c0 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:04.344872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b15c5-dad2-46f9-a69d-7fc76e3849c0 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:04.346176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:54:04.350983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7835ef-5201-4c53-a701-12c6eefaca70 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:04.351420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7835ef-5201-4c53-a701-12c6eefaca70 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:04.351525 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7835ef-5201-4c53-a701-12c6eefaca70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:04.369930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-de7835ef-5201-4c53-a701-12c6eefaca70 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:54:04.371181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-de7835ef-5201-4c53-a701-12c6eefaca70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:54:04.378945 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:54:04.534300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b15c5-dad2-46f9-a69d-7fc76e3849c0 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:54:04.548411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b15c5-dad2-46f9-a69d-7fc76e3849c0 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:54:04.553980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900a9f2-d3e4-4187-ba5b-bcca6ed3248b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:54:04.554606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900a9f2-d3e4-4187-ba5b-bcca6ed3248b service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:04.554887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900a9f2-d3e4-4187-ba5b-bcca6ed3248b service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:04.555162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900a9f2-d3e4-4187-ba5b-bcca6ed3248b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:04.563231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900a9f2-d3e4-4187-ba5b-bcca6ed3248b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:54:04.564819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900a9f2-d3e4-4187-ba5b-bcca6ed3248b service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:54:04.574179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 755/1524] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:54:05.434419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36641bdc-7ab4-41b1-8391-d00e79e995f4 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:05.434988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36641bdc-7ab4-41b1-8391-d00e79e995f4 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:05.434988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36641bdc-7ab4-41b1-8391-d00e79e995f4 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:05.436327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 18:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:54:05.441664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400199-5e44-4085-b6a1-939e3ee57e3a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:05.442117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400199-5e44-4085-b6a1-939e3ee57e3a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:05.442221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a400199-5e44-4085-b6a1-939e3ee57e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:05.459585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-3a400199-5e44-4085-b6a1-939e3ee57e3a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:54:05.460772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-3a400199-5e44-4085-b6a1-939e3ee57e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:54:05.468643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 18:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:54:05.620759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36641bdc-7ab4-41b1-8391-d00e79e995f4 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:54:05.634661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36641bdc-7ab4-41b1-8391-d00e79e995f4 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:54:05.639893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec9e1d-d0a5-46f7-bbe4-fab9760bbfc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:54:05.640524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec9e1d-d0a5-46f7-bbe4-fab9760bbfc9 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:05.640804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec9e1d-d0a5-46f7-bbe4-fab9760bbfc9 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:05.641076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec9e1d-d0a5-46f7-bbe4-fab9760bbfc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:05.648941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec9e1d-d0a5-46f7-bbe4-fab9760bbfc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:54:05.650592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec9e1d-d0a5-46f7-bbe4-fab9760bbfc9 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:54:05.660174 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 771/1527] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:54:09.869134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad91d2-85a1-483b-b3c0-c424170bde91 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:09.869733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad91d2-85a1-483b-b3c0-c424170bde91 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:09.869733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad91d2-85a1-483b-b3c0-c424170bde91 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:09.871091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:54:09.876628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571aaf9-cdf8-47c2-a071-a4e1dc3bc325 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:09.876628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571aaf9-cdf8-47c2-a071-a4e1dc3bc325 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:09.877586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571aaf9-cdf8-47c2-a071-a4e1dc3bc325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:09.895327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-a571aaf9-cdf8-47c2-a071-a4e1dc3bc325 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:54:09.896520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a571aaf9-cdf8-47c2-a071-a4e1dc3bc325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:54:09.904225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:54:09.941647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad91d2-85a1-483b-b3c0-c424170bde91 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:54:09.943412 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ffad91d2-85a1-483b-b3c0-c424170bde91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:54:09.998253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad91d2-85a1-483b-b3c0-c424170bde91 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:54:10.002342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68e47c-cea2-41e3-9937-95e00bc60c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:54:10.002976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68e47c-cea2-41e3-9937-95e00bc60c6e service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:10.003259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68e47c-cea2-41e3-9937-95e00bc60c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:10.003536 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68e47c-cea2-41e3-9937-95e00bc60c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:10.012278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68e47c-cea2-41e3-9937-95e00bc60c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:54:10.013732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68e47c-cea2-41e3-9937-95e00bc60c6e service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:54:10.018534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:54:10.023082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 758/1530] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:54:12.143090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226991c-b937-4270-bdf9-33c0d8e6417f None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:12.143646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226991c-b937-4270-bdf9-33c0d8e6417f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:12.143646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226991c-b937-4270-bdf9-33c0d8e6417f None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:12.145043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:54:12.150558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e4f54-dd95-4a15-86c7-736ab6541d45 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:12.150999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e4f54-dd95-4a15-86c7-736ab6541d45 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:12.151095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e4f54-dd95-4a15-86c7-736ab6541d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:12.168762 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-7d8e4f54-dd95-4a15-86c7-736ab6541d45 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:54:12.170055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-7d8e4f54-dd95-4a15-86c7-736ab6541d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:54:12.178469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:54:12.214023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226991c-b937-4270-bdf9-33c0d8e6417f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:54:12.224425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226991c-b937-4270-bdf9-33c0d8e6417f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:54:12.227828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89410c6e-c5c2-435d-b309-4ca24922813e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:54:12.228453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89410c6e-c5c2-435d-b309-4ca24922813e service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:12.228735 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89410c6e-c5c2-435d-b309-4ca24922813e service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:12.229006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89410c6e-c5c2-435d-b309-4ca24922813e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:12.238330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89410c6e-c5c2-435d-b309-4ca24922813e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:54:12.239740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89410c6e-c5c2-435d-b309-4ca24922813e service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:54:12.248859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 774/1533] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:54:15.887765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70c1c-6ff3-44c4-9144-ef48a458c855 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:15.888202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70c1c-6ff3-44c4-9144-ef48a458c855 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:15.888305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce70c1c-6ff3-44c4-9144-ef48a458c855 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:15.889714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:54:15.895406 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da162beb-a8d1-4e9c-8e52-e2756fe32b61 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:15.895817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da162beb-a8d1-4e9c-8e52-e2756fe32b61 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:15.895959 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da162beb-a8d1-4e9c-8e52-e2756fe32b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:15.914182 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-da162beb-a8d1-4e9c-8e52-e2756fe32b61 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:54:15.918308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-da162beb-a8d1-4e9c-8e52-e2756fe32b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:54:15.923167 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:54:16.193383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce70c1c-6ff3-44c4-9144-ef48a458c855 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:54:16.198178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6ce70c1c-6ff3-44c4-9144-ef48a458c855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:54:16.252209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce70c1c-6ff3-44c4-9144-ef48a458c855 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:54:16.255830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2e607-c95b-47a3-aebe-b0b6aa39abef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:54:16.256474 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2e607-c95b-47a3-aebe-b0b6aa39abef service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:16.256794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2e607-c95b-47a3-aebe-b0b6aa39abef service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:16.257075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2e607-c95b-47a3-aebe-b0b6aa39abef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:16.266099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2e607-c95b-47a3-aebe-b0b6aa39abef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:54:16.267531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2e607-c95b-47a3-aebe-b0b6aa39abef service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:54:16.277008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 761/1536] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:54:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:54:18.402434 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5646ce-1a0a-44c2-bf73-1983039ddf89 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:18.402955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5646ce-1a0a-44c2-bf73-1983039ddf89 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:18.403064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5646ce-1a0a-44c2-bf73-1983039ddf89 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:18.404341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 18:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:54:18.410068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32e5d8-b947-45b2-befd-b54326526ddf None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:18.410523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32e5d8-b947-45b2-befd-b54326526ddf None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:18.410633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32e5d8-b947-45b2-befd-b54326526ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:18.428832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-aa32e5d8-b947-45b2-befd-b54326526ddf None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:54:18.430087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-aa32e5d8-b947-45b2-befd-b54326526ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:54:18.437916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 18:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:54:18.700773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5646ce-1a0a-44c2-bf73-1983039ddf89 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:54:18.702488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5f5646ce-1a0a-44c2-bf73-1983039ddf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:54:18.716055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5646ce-1a0a-44c2-bf73-1983039ddf89 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:54:18.719511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64455e2b-969e-48fe-8978-ad840156c8f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:54:18.720146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64455e2b-969e-48fe-8978-ad840156c8f4 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:54:18.720431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64455e2b-969e-48fe-8978-ad840156c8f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:54:18.720707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64455e2b-969e-48fe-8978-ad840156c8f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:54:18.729515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64455e2b-969e-48fe-8978-ad840156c8f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:54:18.730930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64455e2b-969e-48fe-8978-ad840156c8f4 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:54:18.740619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 777/1539] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:54:28.422973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=32 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:54:28.736061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=70 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:46.706953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639477c-2a79-4467-bebc-2be606030b74 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:46.707557 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639477c-2a79-4467-bebc-2be606030b74 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:46.707713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639477c-2a79-4467-bebc-2be606030b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:46.734230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-0639477c-2a79-4467-bebc-2be606030b74 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:46.736009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0639477c-2a79-4467-bebc-2be606030b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:46.753940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 763/1540] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:46.770903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171825-bf73-4446-9c38-4d4256746319 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:46.771474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171825-bf73-4446-9c38-4d4256746319 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:46.771474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66171825-bf73-4446-9c38-4d4256746319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:46.801262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-66171825-bf73-4446-9c38-4d4256746319 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:46.801262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-66171825-bf73-4446-9c38-4d4256746319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:46.816428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 778/1541] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:46.822476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639477c-2a79-4467-bebc-2be606030b74 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:46.837159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639477c-2a79-4467-bebc-2be606030b74 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:46.844411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9d910-78c4-4af3-b0c5-86d1f97761c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:46.845516 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9d910-78c4-4af3-b0c5-86d1f97761c5 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:46.846023 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9d910-78c4-4af3-b0c5-86d1f97761c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:46.846512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9d910-78c4-4af3-b0c5-86d1f97761c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:46.847493 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9d910-78c4-4af3-b0c5-86d1f97761c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:55:46.848034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9d910-78c4-4af3-b0c5-86d1f97761c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:46.849912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9d910-78c4-4af3-b0c5-86d1f97761c5 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:46.866885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 764/1542] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:55:46.875569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66171825-bf73-4446-9c38-4d4256746319 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:46.891251 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66171825-bf73-4446-9c38-4d4256746319 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:46.906936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce4385-33f3-4060-86d7-d16f7ba7b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:46.907626 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce4385-33f3-4060-86d7-d16f7ba7b451 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:46.907908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce4385-33f3-4060-86d7-d16f7ba7b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:46.908276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce4385-33f3-4060-86d7-d16f7ba7b451 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:46.909007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce4385-33f3-4060-86d7-d16f7ba7b451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:46.912136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ce4385-33f3-4060-86d7-d16f7ba7b451 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:46.996117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 779/1543] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:46 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:47.000190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9d910-78c4-4af3-b0c5-86d1f97761c5 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.016632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9d910-78c4-4af3-b0c5-86d1f97761c5 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:47.102903 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23ae7d-1d1f-48a6-9ef2-0a9402e62af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:47.104025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23ae7d-1d1f-48a6-9ef2-0a9402e62af0 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.104548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23ae7d-1d1f-48a6-9ef2-0a9402e62af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.105041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23ae7d-1d1f-48a6-9ef2-0a9402e62af0 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.106068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23ae7d-1d1f-48a6-9ef2-0a9402e62af0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:47.110227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23ae7d-1d1f-48a6-9ef2-0a9402e62af0 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:47.162975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074809c-fc9c-4bbc-b8f0-75e5837c2a13 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.163271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074809c-fc9c-4bbc-b8f0-75e5837c2a13 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.163559 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0074809c-fc9c-4bbc-b8f0-75e5837c2a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.194486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0074809c-fc9c-4bbc-b8f0-75e5837c2a13 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.196211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0074809c-fc9c-4bbc-b8f0-75e5837c2a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.211680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 780/1544] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:55:47.223744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b818d-0f21-45cd-83cf-e95783197361 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.224339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b818d-0f21-45cd-83cf-e95783197361 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.224838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b818d-0f21-45cd-83cf-e95783197361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.225226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 765/1545] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:46 2022] POST /identity/v3/users => generated 288 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:47.230869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23ae7d-1d1f-48a6-9ef2-0a9402e62af0 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.254642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-707b818d-0f21-45cd-83cf-e95783197361 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.256170 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-707b818d-0f21-45cd-83cf-e95783197361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.268814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23ae7d-1d1f-48a6-9ef2-0a9402e62af0 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:47.272395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 781/1546] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:55:47.277796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f70545-175d-4fc9-a362-2ff868f00147 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.278550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f70545-175d-4fc9-a362-2ff868f00147 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.279075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f70545-175d-4fc9-a362-2ff868f00147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.299547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697265f-e988-4d20-801b-9b751e0a8993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:47.300600 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697265f-e988-4d20-801b-9b751e0a8993 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.301053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697265f-e988-4d20-801b-9b751e0a8993 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.301550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697265f-e988-4d20-801b-9b751e0a8993 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.302468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697265f-e988-4d20-801b-9b751e0a8993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:47.304093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697265f-e988-4d20-801b-9b751e0a8993 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:47.306365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-b7f70545-175d-4fc9-a362-2ff868f00147 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.307994 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b7f70545-175d-4fc9-a362-2ff868f00147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.329915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 782/1547] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:55:47.330789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f70545-175d-4fc9-a362-2ff868f00147 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.331654 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 766/1548] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:55:47.335436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697265f-e988-4d20-801b-9b751e0a8993 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.352821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697265f-e988-4d20-801b-9b751e0a8993 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:47.368747 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f70545-175d-4fc9-a362-2ff868f00147 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:47.370770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39c6c0-d1e0-47cc-b403-df99af72c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:47.371471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39c6c0-d1e0-47cc-b403-df99af72c442 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.371754 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39c6c0-d1e0-47cc-b403-df99af72c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.372025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39c6c0-d1e0-47cc-b403-df99af72c442 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.377575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39c6c0-d1e0-47cc-b403-df99af72c442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:47.379433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39c6c0-d1e0-47cc-b403-df99af72c442 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:47.383611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024ed79-f77a-4fa9-b5e8-ea503b7a1088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:47.384350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024ed79-f77a-4fa9-b5e8-ea503b7a1088 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.385573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024ed79-f77a-4fa9-b5e8-ea503b7a1088 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.385573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024ed79-f77a-4fa9-b5e8-ea503b7a1088 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.386213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024ed79-f77a-4fa9-b5e8-ea503b7a1088 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:55:47.386527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024ed79-f77a-4fa9-b5e8-ea503b7a1088 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:47.387577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: WARNING py.warnings [None req-4024ed79-f77a-4fa9-b5e8-ea503b7a1088 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:47.387577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: warnings.warn(msg) Jun 16 18:55:47.387577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: Jun 16 18:55:47.388899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024ed79-f77a-4fa9-b5e8-ea503b7a1088 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:47.394493 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 767/1549] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:55:47.395839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649fe26-c405-4376-b37a-86419aa8a2cd None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.396133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649fe26-c405-4376-b37a-86419aa8a2cd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.396408 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649fe26-c405-4376-b37a-86419aa8a2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.404043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 783/1550] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:55:47.405411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fc076-19e4-46ce-b243-0d5705ae47a0 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.405713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fc076-19e4-46ce-b243-0d5705ae47a0 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.405977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fc076-19e4-46ce-b243-0d5705ae47a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.434797 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d649fe26-c405-4376-b37a-86419aa8a2cd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.436022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d649fe26-c405-4376-b37a-86419aa8a2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.443611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-200fc076-19e4-46ce-b243-0d5705ae47a0 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.444787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-200fc076-19e4-46ce-b243-0d5705ae47a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.454508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 768/1551] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:47.455781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293ac6c-6c33-4840-b47f-4e8a591af100 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.456069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293ac6c-6c33-4840-b47f-4e8a591af100 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.458393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293ac6c-6c33-4840-b47f-4e8a591af100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.462409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 784/1552] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:47.463200 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fc076-19e4-46ce-b243-0d5705ae47a0 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.476852 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fc076-19e4-46ce-b243-0d5705ae47a0 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:47.481409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f536115-81d6-4f9a-a2b6-eda44852b94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:47.482300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f536115-81d6-4f9a-a2b6-eda44852b94c admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.482585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f536115-81d6-4f9a-a2b6-eda44852b94c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.482863 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f536115-81d6-4f9a-a2b6-eda44852b94c admin admin] PATH_INFO: `/v3/projects/f9e5f71068aa4d75bcc4578b741f490d/users/fec4e3ac2e44403b9d45b37b5a6f64a9/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.487644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-e293ac6c-6c33-4840-b47f-4e8a591af100 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.488857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e293ac6c-6c33-4840-b47f-4e8a591af100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.497993 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f536115-81d6-4f9a-a2b6-eda44852b94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e5f71068aa4d75bcc4578b741f490d, user_id=fec4e3ac2e44403b9d45b37b5a6f64a9, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:47.501397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 769/1553] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:47.501397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293ac6c-6c33-4840-b47f-4e8a591af100 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.501619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f536115-81d6-4f9a-a2b6-eda44852b94c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:47.513032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293ac6c-6c33-4840-b47f-4e8a591af100 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:47.518019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367fe3b-ffcb-449a-a592-0d78d9ba9a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:47.519530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367fe3b-ffcb-449a-a592-0d78d9ba9a47 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.519834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367fe3b-ffcb-449a-a592-0d78d9ba9a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.520113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367fe3b-ffcb-449a-a592-0d78d9ba9a47 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.520785 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367fe3b-ffcb-449a-a592-0d78d9ba9a47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:47.521841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: WARNING py.warnings [None req-0367fe3b-ffcb-449a-a592-0d78d9ba9a47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:47.521841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: warnings.warn(msg) Jun 16 18:55:47.521841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: Jun 16 18:55:47.524477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367fe3b-ffcb-449a-a592-0d78d9ba9a47 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:47.580691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 785/1554] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:47 2022] PUT /identity/v3/projects/f9e5f71068aa4d75bcc4578b741f490d/users/fec4e3ac2e44403b9d45b37b5a6f64a9/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:47.582605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f536115-81d6-4f9a-a2b6-eda44852b94c admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.604917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 770/1555] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:47.606528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa62ed-e128-4ef5-97d6-634a8f802990 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.606824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa62ed-e128-4ef5-97d6-634a8f802990 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.607110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa62ed-e128-4ef5-97d6-634a8f802990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.625942 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-bafa62ed-e128-4ef5-97d6-634a8f802990 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.627675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-bafa62ed-e128-4ef5-97d6-634a8f802990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.645841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f536115-81d6-4f9a-a2b6-eda44852b94c admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:47.685913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 771/1556] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:47.687188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89ee65-2de8-435e-9042-0524c8033d41 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.687598 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89ee65-2de8-435e-9042-0524c8033d41 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.687886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89ee65-2de8-435e-9042-0524c8033d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.705925 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-8b89ee65-2de8-435e-9042-0524c8033d41 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.706103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ac385-3c8e-4e55-84e2-0d52167c97f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:47.706706 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ac385-3c8e-4e55-84e2-0d52167c97f2 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.706997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ac385-3c8e-4e55-84e2-0d52167c97f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.707237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8b89ee65-2de8-435e-9042-0524c8033d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.707716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ac385-3c8e-4e55-84e2-0d52167c97f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.707978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ac385-3c8e-4e55-84e2-0d52167c97f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:55:47.708286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ac385-3c8e-4e55-84e2-0d52167c97f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:47.709915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ac385-3c8e-4e55-84e2-0d52167c97f2 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:47.715150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 772/1557] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:47.716349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617b32d-45ce-46c1-aecf-15b4b6f57364 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.716638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617b32d-45ce-46c1-aecf-15b4b6f57364 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.716912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617b32d-45ce-46c1-aecf-15b4b6f57364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.723988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 786/1558] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:55:47.725112 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38615cbc-29ae-47ff-94f8-9ca1a4ac5b41 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.725452 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38615cbc-29ae-47ff-94f8-9ca1a4ac5b41 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.725730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38615cbc-29ae-47ff-94f8-9ca1a4ac5b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.729487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:47.736054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6617b32d-45ce-46c1-aecf-15b4b6f57364 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.737263 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6617b32d-45ce-46c1-aecf-15b4b6f57364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.745357 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-38615cbc-29ae-47ff-94f8-9ca1a4ac5b41 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.745357 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-38615cbc-29ae-47ff-94f8-9ca1a4ac5b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.745763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 773/1559] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:47.746957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cad2c-c6af-4c44-8e50-3915911d9c85 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.747245 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cad2c-c6af-4c44-8e50-3915911d9c85 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.747572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cad2c-c6af-4c44-8e50-3915911d9c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.754385 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 787/1560] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:47.756534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e095c3-f4d4-49e8-8b63-6ac53969e7f8 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.756828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e095c3-f4d4-49e8-8b63-6ac53969e7f8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.757108 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e095c3-f4d4-49e8-8b63-6ac53969e7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.762803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:47.765157 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-e18cad2c-c6af-4c44-8e50-3915911d9c85 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.766297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e18cad2c-c6af-4c44-8e50-3915911d9c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.775075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-99e095c3-f4d4-49e8-8b63-6ac53969e7f8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.775618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 774/1561] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:47.776260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-99e095c3-f4d4-49e8-8b63-6ac53969e7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.776570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cad2c-c6af-4c44-8e50-3915911d9c85 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.783920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 788/1562] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:47.785753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a5858-f74a-4caf-8174-1e53e8cd4ba9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.786045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a5858-f74a-4caf-8174-1e53e8cd4ba9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.789053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a5858-f74a-4caf-8174-1e53e8cd4ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.789179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cad2c-c6af-4c44-8e50-3915911d9c85 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:47.792213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b820e9a-538a-416d-8955-ad1347ad008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:47.792899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b820e9a-538a-416d-8955-ad1347ad008e admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.793189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b820e9a-538a-416d-8955-ad1347ad008e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.793498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b820e9a-538a-416d-8955-ad1347ad008e admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.794171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b820e9a-538a-416d-8955-ad1347ad008e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:47.796132 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b820e9a-538a-416d-8955-ad1347ad008e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:47.813119 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f85a5858-f74a-4caf-8174-1e53e8cd4ba9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.817330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f85a5858-f74a-4caf-8174-1e53e8cd4ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.822677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 789/1563] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:47.823570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a5858-f74a-4caf-8174-1e53e8cd4ba9 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.832951 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a5858-f74a-4caf-8174-1e53e8cd4ba9 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:47.837385 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a7d25-6c58-4ccf-9cd8-2071c8dd9d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:47.837999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a7d25-6c58-4ccf-9cd8-2071c8dd9d11 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.838285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a7d25-6c58-4ccf-9cd8-2071c8dd9d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.838552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a7d25-6c58-4ccf-9cd8-2071c8dd9d11 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.839244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a7d25-6c58-4ccf-9cd8-2071c8dd9d11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:55:47.839547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a7d25-6c58-4ccf-9cd8-2071c8dd9d11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:47.841107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a7d25-6c58-4ccf-9cd8-2071c8dd9d11 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:47.852576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 790/1564] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:55:47.863183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a7d25-6c58-4ccf-9cd8-2071c8dd9d11 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.863183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a7d25-6c58-4ccf-9cd8-2071c8dd9d11 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:47.867378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620658d-c7dc-42d8-9ebc-8b9e5b84dbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:47.868012 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620658d-c7dc-42d8-9ebc-8b9e5b84dbbe admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.868297 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620658d-c7dc-42d8-9ebc-8b9e5b84dbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.868571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620658d-c7dc-42d8-9ebc-8b9e5b84dbbe admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.869366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620658d-c7dc-42d8-9ebc-8b9e5b84dbbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:47.871239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620658d-c7dc-42d8-9ebc-8b9e5b84dbbe admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:47.894348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 775/1565] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:47.895612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cba5f2-8932-4a9c-81dc-9284e8379836 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.895902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cba5f2-8932-4a9c-81dc-9284e8379836 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.896177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cba5f2-8932-4a9c-81dc-9284e8379836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.913867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-58cba5f2-8932-4a9c-81dc-9284e8379836 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.915055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-58cba5f2-8932-4a9c-81dc-9284e8379836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:47.923189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 776/1566] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:47.923901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cba5f2-8932-4a9c-81dc-9284e8379836 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.934842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cba5f2-8932-4a9c-81dc-9284e8379836 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:47.937541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd0369-58f8-4fdd-b711-cb1d4f89690b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:47.938187 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd0369-58f8-4fdd-b711-cb1d4f89690b admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.938471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd0369-58f8-4fdd-b711-cb1d4f89690b admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.938749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd0369-58f8-4fdd-b711-cb1d4f89690b admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.939486 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd0369-58f8-4fdd-b711-cb1d4f89690b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:55:47.939795 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd0369-58f8-4fdd-b711-cb1d4f89690b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:47.941467 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd0369-58f8-4fdd-b711-cb1d4f89690b admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:47.952870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 777/1567] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:55:47.957886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c4636-2dc1-4495-90a8-323ae433adf4 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:47.957886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c4636-2dc1-4495-90a8-323ae433adf4 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:47.957886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c4636-2dc1-4495-90a8-323ae433adf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:47.958878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 791/1568] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:47.959709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620658d-c7dc-42d8-9ebc-8b9e5b84dbbe admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:47.977057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-600c4636-2dc1-4495-90a8-323ae433adf4 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:47.978332 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-600c4636-2dc1-4495-90a8-323ae433adf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:48.021804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620658d-c7dc-42d8-9ebc-8b9e5b84dbbe admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.029125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbefa-4b67-408d-b5c7-305725952c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.029984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbefa-4b67-408d-b5c7-305725952c17 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.030266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbefa-4b67-408d-b5c7-305725952c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.030545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542cbefa-4b67-408d-b5c7-305725952c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.038927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 778/1569] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:48.039846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c4636-2dc1-4495-90a8-323ae433adf4 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.049418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c4636-2dc1-4495-90a8-323ae433adf4 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.054263 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ce57f-813e-4085-a37a-51c973f64855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.054901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ce57f-813e-4085-a37a-51c973f64855 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.055181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ce57f-813e-4085-a37a-51c973f64855 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.055455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ce57f-813e-4085-a37a-51c973f64855 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.056159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ce57f-813e-4085-a37a-51c973f64855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:55:48.056460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ce57f-813e-4085-a37a-51c973f64855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.058081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ce57f-813e-4085-a37a-51c973f64855 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.070332 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 779/1570] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:55:48.071779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ce57f-813e-4085-a37a-51c973f64855 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.082372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ce57f-813e-4085-a37a-51c973f64855 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.087364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb41c3a-bdc6-4410-8c2f-646b5d093793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.088047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb41c3a-bdc6-4410-8c2f-646b5d093793 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.088334 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb41c3a-bdc6-4410-8c2f-646b5d093793 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.088616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb41c3a-bdc6-4410-8c2f-646b5d093793 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.089340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb41c3a-bdc6-4410-8c2f-646b5d093793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.091493 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb41c3a-bdc6-4410-8c2f-646b5d093793 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.101115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cbefa-4b67-408d-b5c7-305725952c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.102586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542cbefa-4b67-408d-b5c7-305725952c17 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.112532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 792/1571] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:55:48.113458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbefa-4b67-408d-b5c7-305725952c17 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.122505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542cbefa-4b67-408d-b5c7-305725952c17 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.127459 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392b617-fd8a-4bde-a111-73c95843a51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.128494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392b617-fd8a-4bde-a111-73c95843a51a admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.128785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392b617-fd8a-4bde-a111-73c95843a51a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.129058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392b617-fd8a-4bde-a111-73c95843a51a admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.129811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392b617-fd8a-4bde-a111-73c95843a51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.131386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392b617-fd8a-4bde-a111-73c95843a51a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.141375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 793/1572] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:48.142164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392b617-fd8a-4bde-a111-73c95843a51a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.151697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392b617-fd8a-4bde-a111-73c95843a51a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.156389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c894a-5af9-42e1-b616-4b46ac0bb60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.157009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c894a-5af9-42e1-b616-4b46ac0bb60c admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.157348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c894a-5af9-42e1-b616-4b46ac0bb60c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.157625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c894a-5af9-42e1-b616-4b46ac0bb60c admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.158458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c894a-5af9-42e1-b616-4b46ac0bb60c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:55:48.158816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c894a-5af9-42e1-b616-4b46ac0bb60c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.160966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c894a-5af9-42e1-b616-4b46ac0bb60c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.172876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 794/1573] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:55:48.176853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c894a-5af9-42e1-b616-4b46ac0bb60c admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.183663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c894a-5af9-42e1-b616-4b46ac0bb60c admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.188147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791957a-710b-404f-9696-d421765c91c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.188782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791957a-710b-404f-9696-d421765c91c7 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.189063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791957a-710b-404f-9696-d421765c91c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.189375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791957a-710b-404f-9696-d421765c91c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.190041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791957a-710b-404f-9696-d421765c91c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.191993 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791957a-710b-404f-9696-d421765c91c7 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.200455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 780/1574] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:48 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:48.201455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb41c3a-bdc6-4410-8c2f-646b5d093793 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.264321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb41c3a-bdc6-4410-8c2f-646b5d093793 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.269016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82c8cc-299d-4da8-a76e-9628d3103fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.269719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82c8cc-299d-4da8-a76e-9628d3103fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.270002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82c8cc-299d-4da8-a76e-9628d3103fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.279713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82c8cc-299d-4da8-a76e-9628d3103fb4 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.279713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82c8cc-299d-4da8-a76e-9628d3103fb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.279713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82c8cc-299d-4da8-a76e-9628d3103fb4 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.302718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 795/1575] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:48 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:48.303555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791957a-710b-404f-9696-d421765c91c7 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.322339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:48.357032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 781/1576] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:48 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:48.357991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82c8cc-299d-4da8-a76e-9628d3103fb4 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.365351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791957a-710b-404f-9696-d421765c91c7 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.369383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82c8cc-299d-4da8-a76e-9628d3103fb4 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.370639 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890ae71-1b9e-454b-9037-9d5e5cf40768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.371311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890ae71-1b9e-454b-9037-9d5e5cf40768 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.371696 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890ae71-1b9e-454b-9037-9d5e5cf40768 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.371980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890ae71-1b9e-454b-9037-9d5e5cf40768 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.372707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890ae71-1b9e-454b-9037-9d5e5cf40768 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:55:48.373020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890ae71-1b9e-454b-9037-9d5e5cf40768 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.374008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58100f0-eb1c-4f12-b6da-118b3098903a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.374657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58100f0-eb1c-4f12-b6da-118b3098903a admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.374768 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890ae71-1b9e-454b-9037-9d5e5cf40768 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.374943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58100f0-eb1c-4f12-b6da-118b3098903a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.375220 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58100f0-eb1c-4f12-b6da-118b3098903a admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.375894 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58100f0-eb1c-4f12-b6da-118b3098903a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.377818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58100f0-eb1c-4f12-b6da-118b3098903a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.386641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:48.397931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 796/1577] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:55:48.397931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890ae71-1b9e-454b-9037-9d5e5cf40768 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.404568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890ae71-1b9e-454b-9037-9d5e5cf40768 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.410042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37484fe-5ee5-41d1-982b-5fde260f482f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.410710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37484fe-5ee5-41d1-982b-5fde260f482f admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.410994 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37484fe-5ee5-41d1-982b-5fde260f482f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.411271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37484fe-5ee5-41d1-982b-5fde260f482f admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.412005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37484fe-5ee5-41d1-982b-5fde260f482f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.413819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37484fe-5ee5-41d1-982b-5fde260f482f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.427110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 797/1578] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:48.427110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37484fe-5ee5-41d1-982b-5fde260f482f admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.434582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37484fe-5ee5-41d1-982b-5fde260f482f admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.439225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5176545-d1ce-4700-a210-51d33e8bd6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.439947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5176545-d1ce-4700-a210-51d33e8bd6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.440236 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5176545-d1ce-4700-a210-51d33e8bd6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.440567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5176545-d1ce-4700-a210-51d33e8bd6f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.441224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5176545-d1ce-4700-a210-51d33e8bd6f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.446231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5176545-d1ce-4700-a210-51d33e8bd6f4 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.497249 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 782/1579] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:48 2022] POST /identity/v3/projects => generated 405 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:48.498077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58100f0-eb1c-4f12-b6da-118b3098903a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.551257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 798/1580] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:48 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:48.551257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5176545-d1ce-4700-a210-51d33e8bd6f4 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.566255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58100f0-eb1c-4f12-b6da-118b3098903a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.621830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5176545-d1ce-4700-a210-51d33e8bd6f4 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.627442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c411ff-f060-4e78-9ec9-a01ef4df5990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.629089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c411ff-f060-4e78-9ec9-a01ef4df5990 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.629412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c411ff-f060-4e78-9ec9-a01ef4df5990 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.634888 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c411ff-f060-4e78-9ec9-a01ef4df5990 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.634996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f4536-c165-43cd-a538-72cb0cc8a022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.635107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c411ff-f060-4e78-9ec9-a01ef4df5990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.635196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f4536-c165-43cd-a538-72cb0cc8a022 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.635196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f4536-c165-43cd-a538-72cb0cc8a022 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.635350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: WARNING py.warnings [None req-d6c411ff-f060-4e78-9ec9-a01ef4df5990 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:48.635350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: warnings.warn(msg) Jun 16 18:55:48.635350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: Jun 16 18:55:48.635578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f4536-c165-43cd-a538-72cb0cc8a022 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.635578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f4536-c165-43cd-a538-72cb0cc8a022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.635728 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c411ff-f060-4e78-9ec9-a01ef4df5990 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.635809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f4536-c165-43cd-a538-72cb0cc8a022 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.724292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 799/1581] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:48 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:48.725428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c411ff-f060-4e78-9ec9-a01ef4df5990 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.735666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c411ff-f060-4e78-9ec9-a01ef4df5990 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.740988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a95036-d7f4-4f49-9503-7f716b96a96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.741698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a95036-d7f4-4f49-9503-7f716b96a96e admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.741982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a95036-d7f4-4f49-9503-7f716b96a96e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.742261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a95036-d7f4-4f49-9503-7f716b96a96e admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.742988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a95036-d7f4-4f49-9503-7f716b96a96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.744591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a95036-d7f4-4f49-9503-7f716b96a96e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.754762 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 783/1582] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:48 2022] POST /identity/v3/users => generated 291 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:48.756017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 800/1583] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:48.756134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f4536-c165-43cd-a538-72cb0cc8a022 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.759741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a95036-d7f4-4f49-9503-7f716b96a96e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.767599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f4536-c165-43cd-a538-72cb0cc8a022 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.769441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a95036-d7f4-4f49-9503-7f716b96a96e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.772353 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d02a2e-a77b-4348-9514-281228e20b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.773049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d02a2e-a77b-4348-9514-281228e20b6e admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.773374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d02a2e-a77b-4348-9514-281228e20b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.773652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d02a2e-a77b-4348-9514-281228e20b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.774309 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078eea7-0816-48bc-bacc-3c6a12699eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:48.774442 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d02a2e-a77b-4348-9514-281228e20b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.775250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078eea7-0816-48bc-bacc-3c6a12699eba admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:48.775707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078eea7-0816-48bc-bacc-3c6a12699eba admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:48.775926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078eea7-0816-48bc-bacc-3c6a12699eba admin admin] PATH_INFO: `/v3/projects/6c22b67eb39f4392baa0b135b7f58dae/users/6367ef477233433a9fab0dab1dd734f7/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:48.776281 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d02a2e-a77b-4348-9514-281228e20b6e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.793565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078eea7-0816-48bc-bacc-3c6a12699eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c22b67eb39f4392baa0b135b7f58dae, user_id=6367ef477233433a9fab0dab1dd734f7, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:48.796214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078eea7-0816-48bc-bacc-3c6a12699eba admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:48.893919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 784/1584] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:48 2022] POST /identity/v3/projects => generated 393 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:48.893919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d02a2e-a77b-4348-9514-281228e20b6e admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.907040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 801/1585] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:48 2022] PUT /identity/v3/projects/6c22b67eb39f4392baa0b135b7f58dae/users/6367ef477233433a9fab0dab1dd734f7/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:48.908162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078eea7-0816-48bc-bacc-3c6a12699eba admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:48.956676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:48.967282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d02a2e-a77b-4348-9514-281228e20b6e admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:48.977021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:48.990952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078eea7-0816-48bc-bacc-3c6a12699eba admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.014501 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6405c-1790-4c53-af8d-290bf99e0913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.015186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6405c-1790-4c53-af8d-290bf99e0913 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.015464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6405c-1790-4c53-af8d-290bf99e0913 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.015735 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6405c-1790-4c53-af8d-290bf99e0913 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.016425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6405c-1790-4c53-af8d-290bf99e0913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.018877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6405c-1790-4c53-af8d-290bf99e0913 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.030426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc6700-7b38-46ea-85ec-9c98790042d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.031418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc6700-7b38-46ea-85ec-9c98790042d9 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.031866 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc6700-7b38-46ea-85ec-9c98790042d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.032358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc6700-7b38-46ea-85ec-9c98790042d9 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.033053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc6700-7b38-46ea-85ec-9c98790042d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.035638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc6700-7b38-46ea-85ec-9c98790042d9 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.093673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 802/1586] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:48 2022] POST /identity/v3/users => generated 303 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:49.094692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6405c-1790-4c53-af8d-290bf99e0913 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.104448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6405c-1790-4c53-af8d-290bf99e0913 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.108971 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb10f22-5013-4496-b261-c23fe619505b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.109889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb10f22-5013-4496-b261-c23fe619505b admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.110180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb10f22-5013-4496-b261-c23fe619505b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.110456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb10f22-5013-4496-b261-c23fe619505b admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.111162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb10f22-5013-4496-b261-c23fe619505b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.113390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb10f22-5013-4496-b261-c23fe619505b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.123013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 803/1587] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:49.130755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb10f22-5013-4496-b261-c23fe619505b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.133906 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb10f22-5013-4496-b261-c23fe619505b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.138365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6f9ba-46a6-459c-9431-72934961bc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.138977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6f9ba-46a6-459c-9431-72934961bc1e admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.139260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6f9ba-46a6-459c-9431-72934961bc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.139537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6f9ba-46a6-459c-9431-72934961bc1e admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.140224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6f9ba-46a6-459c-9431-72934961bc1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.141800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6f9ba-46a6-459c-9431-72934961bc1e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.153447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 785/1588] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:48 2022] POST /identity/v3/users => generated 303 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:49.153447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc6700-7b38-46ea-85ec-9c98790042d9 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.159948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 804/1589] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:49.159948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6f9ba-46a6-459c-9431-72934961bc1e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.165629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc6700-7b38-46ea-85ec-9c98790042d9 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.170316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e996fc8-e770-43ac-8c13-be575154c573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.170522 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6f9ba-46a6-459c-9431-72934961bc1e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.171115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e996fc8-e770-43ac-8c13-be575154c573 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.171480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e996fc8-e770-43ac-8c13-be575154c573 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.171766 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e996fc8-e770-43ac-8c13-be575154c573 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.172550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e996fc8-e770-43ac-8c13-be575154c573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.174127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e996fc8-e770-43ac-8c13-be575154c573 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.175599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8d279-b6ea-4c53-8229-00d02fb44eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.176373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8d279-b6ea-4c53-8229-00d02fb44eae admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.176630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8d279-b6ea-4c53-8229-00d02fb44eae admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.176930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8d279-b6ea-4c53-8229-00d02fb44eae admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.177613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8d279-b6ea-4c53-8229-00d02fb44eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.183803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8d279-b6ea-4c53-8229-00d02fb44eae admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.185270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 786/1590] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:49.186529 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416abab0-8ec5-49b5-a2e7-e86cafeb3e5c None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.186825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416abab0-8ec5-49b5-a2e7-e86cafeb3e5c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.187103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416abab0-8ec5-49b5-a2e7-e86cafeb3e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.214091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-416abab0-8ec5-49b5-a2e7-e86cafeb3e5c None None] MFA Rules not processed for user `6367ef477233433a9fab0dab1dd734f7`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:49.215365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-416abab0-8ec5-49b5-a2e7-e86cafeb3e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:49.276641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 805/1591] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:49 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:49.276641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8d279-b6ea-4c53-8229-00d02fb44eae admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.283609 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 787/1592] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:49 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:49.284507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416abab0-8ec5-49b5-a2e7-e86cafeb3e5c None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.290640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8d279-b6ea-4c53-8229-00d02fb44eae admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.294604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416abab0-8ec5-49b5-a2e7-e86cafeb3e5c None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.295525 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e8df8-3b69-4b50-afba-bcebbf276378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.296197 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e8df8-3b69-4b50-afba-bcebbf276378 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.296605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e8df8-3b69-4b50-afba-bcebbf276378 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.296977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e8df8-3b69-4b50-afba-bcebbf276378 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.297815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e8df8-3b69-4b50-afba-bcebbf276378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.299161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfee0c9-796a-4467-8a25-6bb8607c2e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.299544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e8df8-3b69-4b50-afba-bcebbf276378 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.300335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfee0c9-796a-4467-8a25-6bb8607c2e9b admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.300621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfee0c9-796a-4467-8a25-6bb8607c2e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.300899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfee0c9-796a-4467-8a25-6bb8607c2e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.301643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfee0c9-796a-4467-8a25-6bb8607c2e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.303217 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfee0c9-796a-4467-8a25-6bb8607c2e9b admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.310749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 806/1593] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:49.311562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e8df8-3b69-4b50-afba-bcebbf276378 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.316068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 788/1594] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:49.316068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfee0c9-796a-4467-8a25-6bb8607c2e9b admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.322050 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e8df8-3b69-4b50-afba-bcebbf276378 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.323536 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfee0c9-796a-4467-8a25-6bb8607c2e9b admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.326660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6869ba-4d57-420a-830a-0dc6ebde0ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.327289 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6869ba-4d57-420a-830a-0dc6ebde0ffd admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.327572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6869ba-4d57-420a-830a-0dc6ebde0ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.327845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6869ba-4d57-420a-830a-0dc6ebde0ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.328055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb238a-de84-4f78-9590-4eea83c928da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.328556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6869ba-4d57-420a-830a-0dc6ebde0ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.328672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb238a-de84-4f78-9590-4eea83c928da admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.328964 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb238a-de84-4f78-9590-4eea83c928da admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.329294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb238a-de84-4f78-9590-4eea83c928da admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.329972 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb238a-de84-4f78-9590-4eea83c928da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.330158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6869ba-4d57-420a-830a-0dc6ebde0ffd admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.332194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cb238a-de84-4f78-9590-4eea83c928da admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.345488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 807/1595] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:49.345690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 789/1596] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:49.345982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb238a-de84-4f78-9590-4eea83c928da admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.346129 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6869ba-4d57-420a-830a-0dc6ebde0ffd admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.356353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6869ba-4d57-420a-830a-0dc6ebde0ffd admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.360108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb238a-de84-4f78-9590-4eea83c928da admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.361975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d58635-0900-46af-a737-c9188b3b2d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.362130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79e3df-142c-41b1-93c4-ef11bebe800f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.363467 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79e3df-142c-41b1-93c4-ef11bebe800f admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.363584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58635-0900-46af-a737-c9188b3b2d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.363773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58635-0900-46af-a737-c9188b3b2d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.363889 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79e3df-142c-41b1-93c4-ef11bebe800f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.364108 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d58635-0900-46af-a737-c9188b3b2d94 admin admin] PATH_INFO: `/v3/projects/57f75a8c58e340a1bba335f7463c03a1/users/1052dda92d1a4597be43a7acdcd06905/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.364221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79e3df-142c-41b1-93c4-ef11bebe800f admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.364902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79e3df-142c-41b1-93c4-ef11bebe800f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.366665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79e3df-142c-41b1-93c4-ef11bebe800f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.377481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 790/1597] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:49.378381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79e3df-142c-41b1-93c4-ef11bebe800f admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.382713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d58635-0900-46af-a737-c9188b3b2d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f75a8c58e340a1bba335f7463c03a1, user_id=1052dda92d1a4597be43a7acdcd06905, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.385494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d58635-0900-46af-a737-c9188b3b2d94 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.388812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79e3df-142c-41b1-93c4-ef11bebe800f admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.392512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10887-c8e0-49ca-985c-2027a1562cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.393140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10887-c8e0-49ca-985c-2027a1562cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.393975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10887-c8e0-49ca-985c-2027a1562cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.394251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b10887-c8e0-49ca-985c-2027a1562cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.394924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b10887-c8e0-49ca-985c-2027a1562cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.396875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b10887-c8e0-49ca-985c-2027a1562cc6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.449211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 808/1598] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:49 2022] PUT /identity/v3/projects/57f75a8c58e340a1bba335f7463c03a1/users/1052dda92d1a4597be43a7acdcd06905/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:49.450880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d58635-0900-46af-a737-c9188b3b2d94 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.490074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 791/1599] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:49 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:49.490074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10887-c8e0-49ca-985c-2027a1562cc6 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.519397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:49.556750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d58635-0900-46af-a737-c9188b3b2d94 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.562139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b2aff-a846-4546-9daa-50b483998955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.563010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b2aff-a846-4546-9daa-50b483998955 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.563397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b2aff-a846-4546-9daa-50b483998955 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.563775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b2aff-a846-4546-9daa-50b483998955 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.563883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b10887-c8e0-49ca-985c-2027a1562cc6 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.564675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b2aff-a846-4546-9daa-50b483998955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.566608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b2aff-a846-4546-9daa-50b483998955 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.569299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f693a0-6e7f-4a1e-b42a-6f967ecb7001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.570292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f693a0-6e7f-4a1e-b42a-6f967ecb7001 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.570650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f693a0-6e7f-4a1e-b42a-6f967ecb7001 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.570991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f693a0-6e7f-4a1e-b42a-6f967ecb7001 admin admin] PATH_INFO: `/v3/projects/0b607bd5abec4c72842e250a639f99f1/users/2de523abbc7b4f8b972dc97b8a013237/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.580903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 809/1600] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:49.580903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b2aff-a846-4546-9daa-50b483998955 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.587915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f693a0-6e7f-4a1e-b42a-6f967ecb7001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b607bd5abec4c72842e250a639f99f1, user_id=2de523abbc7b4f8b972dc97b8a013237, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.589053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: WARNING py.warnings [None req-55f693a0-6e7f-4a1e-b42a-6f967ecb7001 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:49.589053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: warnings.warn(msg) Jun 16 18:55:49.589053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: Jun 16 18:55:49.591158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b2aff-a846-4546-9daa-50b483998955 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.591374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f693a0-6e7f-4a1e-b42a-6f967ecb7001 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.597178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb97aa-047f-452c-96f2-6be5a4b23e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.597964 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb97aa-047f-452c-96f2-6be5a4b23e3c admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.598306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb97aa-047f-452c-96f2-6be5a4b23e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.598579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb97aa-047f-452c-96f2-6be5a4b23e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.599314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb97aa-047f-452c-96f2-6be5a4b23e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.600950 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb97aa-047f-452c-96f2-6be5a4b23e3c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.611190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 810/1601] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:49.611965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb97aa-047f-452c-96f2-6be5a4b23e3c admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.622216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb97aa-047f-452c-96f2-6be5a4b23e3c admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.627853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535edf84-83bd-4e3f-921c-1a3c8abf9529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.628851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535edf84-83bd-4e3f-921c-1a3c8abf9529 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.629010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535edf84-83bd-4e3f-921c-1a3c8abf9529 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.629308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535edf84-83bd-4e3f-921c-1a3c8abf9529 admin admin] PATH_INFO: `/v3/projects/93ca6ea99cdb4f74b3372635fb2858ba/users/f81961656d094e8aa763228a335f5733/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.644974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535edf84-83bd-4e3f-921c-1a3c8abf9529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93ca6ea99cdb4f74b3372635fb2858ba, user_id=f81961656d094e8aa763228a335f5733, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.650149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535edf84-83bd-4e3f-921c-1a3c8abf9529 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.667376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 792/1602] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:49 2022] PUT /identity/v3/projects/0b607bd5abec4c72842e250a639f99f1/users/2de523abbc7b4f8b972dc97b8a013237/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:49.671954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f693a0-6e7f-4a1e-b42a-6f967ecb7001 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.718179 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:49.734970 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 811/1603] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:49 2022] PUT /identity/v3/projects/93ca6ea99cdb4f74b3372635fb2858ba/users/f81961656d094e8aa763228a335f5733/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:49.736461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336b949-ca86-4e15-893d-e016c9cf771d None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.736752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336b949-ca86-4e15-893d-e016c9cf771d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.737080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336b949-ca86-4e15-893d-e016c9cf771d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.744783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f693a0-6e7f-4a1e-b42a-6f967ecb7001 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.755584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-1336b949-ca86-4e15-893d-e016c9cf771d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:49.756553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-1336b949-ca86-4e15-893d-e016c9cf771d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:49.804787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bed442-5994-47cb-946b-1cb26e5973b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.804787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bed442-5994-47cb-946b-1cb26e5973b4 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.805456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bed442-5994-47cb-946b-1cb26e5973b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.805712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bed442-5994-47cb-946b-1cb26e5973b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.806542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bed442-5994-47cb-946b-1cb26e5973b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.808265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bed442-5994-47cb-946b-1cb26e5973b4 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.819320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 793/1604] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:49.820384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bed442-5994-47cb-946b-1cb26e5973b4 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.828465 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 812/1605] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:49.832117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336b949-ca86-4e15-893d-e016c9cf771d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.832242 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bed442-5994-47cb-946b-1cb26e5973b4 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.841043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336b949-ca86-4e15-893d-e016c9cf771d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.842032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82701402-7e8f-42f1-8f46-66d9ed0e4150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.842919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82701402-7e8f-42f1-8f46-66d9ed0e4150 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.843303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82701402-7e8f-42f1-8f46-66d9ed0e4150 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.843685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82701402-7e8f-42f1-8f46-66d9ed0e4150 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.844477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82701402-7e8f-42f1-8f46-66d9ed0e4150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.846332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bee8d9-307f-49a2-bfe9-4dedf3f5bc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:49.847306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82701402-7e8f-42f1-8f46-66d9ed0e4150 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.848251 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee8d9-307f-49a2-bfe9-4dedf3f5bc34 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.848540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee8d9-307f-49a2-bfe9-4dedf3f5bc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.849116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bee8d9-307f-49a2-bfe9-4dedf3f5bc34 admin admin] PATH_INFO: `/v3/projects/061a027ac5464038b15bbca6c47e7f7c/users/4dc5d1e88e3a45588f671842e8cf9ffd/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.865903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bee8d9-307f-49a2-bfe9-4dedf3f5bc34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061a027ac5464038b15bbca6c47e7f7c, user_id=4dc5d1e88e3a45588f671842e8cf9ffd, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:49.868590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bee8d9-307f-49a2-bfe9-4dedf3f5bc34 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:49.952164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 794/1606] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:49 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:49.952662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 813/1607] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:49 2022] PUT /identity/v3/projects/061a027ac5464038b15bbca6c47e7f7c/users/4dc5d1e88e3a45588f671842e8cf9ffd/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:49.954163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe604f-f6e0-42b6-8e52-7c698cc0ea0f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:49.954524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe604f-f6e0-42b6-8e52-7c698cc0ea0f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:49.955034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe604f-f6e0-42b6-8e52-7c698cc0ea0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:49.957517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82701402-7e8f-42f1-8f46-66d9ed0e4150 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:49.984523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82701402-7e8f-42f1-8f46-66d9ed0e4150 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:49.985479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-09fe604f-f6e0-42b6-8e52-7c698cc0ea0f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:49.986705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-09fe604f-f6e0-42b6-8e52-7c698cc0ea0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:50.023489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:50.056893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38378b0-a625-4ef2-88db-0ea71bf8c8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:50.058743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38378b0-a625-4ef2-88db-0ea71bf8c8b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.061388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38378b0-a625-4ef2-88db-0ea71bf8c8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.061388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38378b0-a625-4ef2-88db-0ea71bf8c8b7 admin admin] PATH_INFO: `/v3/projects/dd6daa05f309410dbe5156c6e55fb175/users/4ea14c9ded8e49cb8f6934d185831ca4/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.080184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38378b0-a625-4ef2-88db-0ea71bf8c8b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd6daa05f309410dbe5156c6e55fb175, user_id=4ea14c9ded8e49cb8f6934d185831ca4, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:50.081537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 814/1608] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:50.082766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef307440-4f27-4a1a-963c-fb4cd4c8c42b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.083157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef307440-4f27-4a1a-963c-fb4cd4c8c42b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.083713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef307440-4f27-4a1a-963c-fb4cd4c8c42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.092893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38378b0-a625-4ef2-88db-0ea71bf8c8b7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:50.116945 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ef307440-4f27-4a1a-963c-fb4cd4c8c42b None None] MFA Rules not processed for user `f81961656d094e8aa763228a335f5733`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:50.118381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ef307440-4f27-4a1a-963c-fb4cd4c8c42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:50.163793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 795/1609] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:49 2022] PUT /identity/v3/projects/dd6daa05f309410dbe5156c6e55fb175/users/4ea14c9ded8e49cb8f6934d185831ca4/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:50.174389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38378b0-a625-4ef2-88db-0ea71bf8c8b7 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:50.192731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 815/1610] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:50.193877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef307440-4f27-4a1a-963c-fb4cd4c8c42b None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:50.200281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:50.245290 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38378b0-a625-4ef2-88db-0ea71bf8c8b7 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:50.252149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e5b77-e7b3-44bd-82bd-aa1759cbd89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:50.253631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e5b77-e7b3-44bd-82bd-aa1759cbd89a admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.261086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e5b77-e7b3-44bd-82bd-aa1759cbd89a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.261086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e5b77-e7b3-44bd-82bd-aa1759cbd89a admin admin] PATH_INFO: `/v3/projects/ed8501d1ff734054ba0a6ef4c398d051/users/7310c631282c495a83e17dbdbc6d2ba0/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.277168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef307440-4f27-4a1a-963c-fb4cd4c8c42b None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:50.281685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e5b77-e7b3-44bd-82bd-aa1759cbd89a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed8501d1ff734054ba0a6ef4c398d051, user_id=7310c631282c495a83e17dbdbc6d2ba0, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:50.281685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e5b77-e7b3-44bd-82bd-aa1759cbd89a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:50.338064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e5dce-8886-4c66-9da2-e0918ed93bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:50.338740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5dce-8886-4c66-9da2-e0918ed93bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.339139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5dce-8886-4c66-9da2-e0918ed93bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.339388 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e5dce-8886-4c66-9da2-e0918ed93bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.350599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e5dce-8886-4c66-9da2-e0918ed93bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:50.352071 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e5dce-8886-4c66-9da2-e0918ed93bb7 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:50.373574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 816/1611] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:55:50.373574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e5dce-8886-4c66-9da2-e0918ed93bb7 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:50.383270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 796/1612] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:50 2022] PUT /identity/v3/projects/ed8501d1ff734054ba0a6ef4c398d051/users/7310c631282c495a83e17dbdbc6d2ba0/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:50.385074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9f00-ced1-4fb7-a223-6dee9caeff51 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.385402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9f00-ced1-4fb7-a223-6dee9caeff51 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.385681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9f00-ced1-4fb7-a223-6dee9caeff51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.388348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e5dce-8886-4c66-9da2-e0918ed93bb7 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:50.414731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-74ee9f00-ced1-4fb7-a223-6dee9caeff51 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:50.416064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-74ee9f00-ced1-4fb7-a223-6dee9caeff51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:50.465715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9e762-4fac-4a55-91e1-dfdfbb7c5529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:50.465715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9e762-4fac-4a55-91e1-dfdfbb7c5529 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.465715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9e762-4fac-4a55-91e1-dfdfbb7c5529 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.465715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9e762-4fac-4a55-91e1-dfdfbb7c5529 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.466264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9e762-4fac-4a55-91e1-dfdfbb7c5529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:50.469334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9e762-4fac-4a55-91e1-dfdfbb7c5529 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:50.562175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 817/1613] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:50.562175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9e762-4fac-4a55-91e1-dfdfbb7c5529 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:50.563531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 797/1614] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:50.563531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdf787-34a4-4b7d-83c2-9b5562687d41 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.563531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdf787-34a4-4b7d-83c2-9b5562687d41 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.563531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bdf787-34a4-4b7d-83c2-9b5562687d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.580428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9e762-4fac-4a55-91e1-dfdfbb7c5529 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:50.589952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02147a-f7e0-4202-8c96-6e432c583f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:50.590466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02147a-f7e0-4202-8c96-6e432c583f1a service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.590667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b8bdf787-34a4-4b7d-83c2-9b5562687d41 None None] MFA Rules not processed for user `4ea14c9ded8e49cb8f6934d185831ca4`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:50.590784 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02147a-f7e0-4202-8c96-6e432c583f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.591013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02147a-f7e0-4202-8c96-6e432c583f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.592712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b8bdf787-34a4-4b7d-83c2-9b5562687d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:50.608875 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02147a-f7e0-4202-8c96-6e432c583f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:50.610608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02147a-f7e0-4202-8c96-6e432c583f1a service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:50.637408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 818/1615] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:55:50.638320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02147a-f7e0-4202-8c96-6e432c583f1a service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:50.648615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:50.651961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:50.658456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02147a-f7e0-4202-8c96-6e432c583f1a service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:50.674201 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fd27d-1cbe-4453-8af6-f6c1bb2bc8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:50.676941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fd27d-1cbe-4453-8af6-f6c1bb2bc8d8 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.676941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fd27d-1cbe-4453-8af6-f6c1bb2bc8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.676941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fd27d-1cbe-4453-8af6-f6c1bb2bc8d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.677713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fd27d-1cbe-4453-8af6-f6c1bb2bc8d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:50.680132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89fd27d-1cbe-4453-8af6-f6c1bb2bc8d8 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:50.706710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 798/1616] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:50 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:50.706710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fae81f-98a3-42ac-9d64-f91464c748fd None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.706710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fae81f-98a3-42ac-9d64-f91464c748fd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.706710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fae81f-98a3-42ac-9d64-f91464c748fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.728940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-07fae81f-98a3-42ac-9d64-f91464c748fd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:50.730205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-07fae81f-98a3-42ac-9d64-f91464c748fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:50.739249 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 799/1617] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:50.743284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fae81f-98a3-42ac-9d64-f91464c748fd None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:50.753066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fae81f-98a3-42ac-9d64-f91464c748fd None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:50.757723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1536b-adeb-4796-ac88-90d5fcfa4921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:50.758666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1536b-adeb-4796-ac88-90d5fcfa4921 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.759002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1536b-adeb-4796-ac88-90d5fcfa4921 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.759282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1536b-adeb-4796-ac88-90d5fcfa4921 admin admin] PATH_INFO: `/v3/projects/dc00b27d9949434189d02d2c078fa22f/users/0f7cca3c7f2e48608d5c31ccde963c39/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.776657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1536b-adeb-4796-ac88-90d5fcfa4921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc00b27d9949434189d02d2c078fa22f, user_id=0f7cca3c7f2e48608d5c31ccde963c39, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:50.784745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1536b-adeb-4796-ac88-90d5fcfa4921 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:50.811034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 819/1618] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:50 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:50.812273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fd27d-1cbe-4453-8af6-f6c1bb2bc8d8 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:50.815496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c89fd27d-1cbe-4453-8af6-f6c1bb2bc8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:50.876210 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fd27d-1cbe-4453-8af6-f6c1bb2bc8d8 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:50.880681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c9859-60ef-442f-9fe8-537d273531ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:50.881586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c9859-60ef-442f-9fe8-537d273531ca service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.882046 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c9859-60ef-442f-9fe8-537d273531ca service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.882403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c9859-60ef-442f-9fe8-537d273531ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.895401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 800/1619] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:50 2022] PUT /identity/v3/projects/dc00b27d9949434189d02d2c078fa22f/users/0f7cca3c7f2e48608d5c31ccde963c39/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:50.896852 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1536b-adeb-4796-ac88-90d5fcfa4921 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:50.967311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c9859-60ef-442f-9fe8-537d273531ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:50.968737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c9859-60ef-442f-9fe8-537d273531ca service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:50.974248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1536b-adeb-4796-ac88-90d5fcfa4921 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:50.983080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 820/1620] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:55:50.983671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b590568-6a8f-462f-8dbf-b27ffaeec3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:50.984533 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c9859-60ef-442f-9fe8-537d273531ca service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:50.984916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b590568-6a8f-462f-8dbf-b27ffaeec3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:50.985380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b590568-6a8f-462f-8dbf-b27ffaeec3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:50.985731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b590568-6a8f-462f-8dbf-b27ffaeec3a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:50.986612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b590568-6a8f-462f-8dbf-b27ffaeec3a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:50.989583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b590568-6a8f-462f-8dbf-b27ffaeec3a5 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:51.014155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c9859-60ef-442f-9fe8-537d273531ca service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:51.099973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a14918-cf63-496b-98cd-240d9f7377d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:51.099973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a14918-cf63-496b-98cd-240d9f7377d0 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:51.099973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a14918-cf63-496b-98cd-240d9f7377d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:51.100383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a14918-cf63-496b-98cd-240d9f7377d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:51.103686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 801/1621] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:50 2022] POST /identity/v3/projects => generated 401 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:51.104970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b590568-6a8f-462f-8dbf-b27ffaeec3a5 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:51.128153 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a14918-cf63-496b-98cd-240d9f7377d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:51.130427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a14918-cf63-496b-98cd-240d9f7377d0 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:51.147747 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 821/1622] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:55:51.148724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a14918-cf63-496b-98cd-240d9f7377d0 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:51.158779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a14918-cf63-496b-98cd-240d9f7377d0 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:51.219162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:51.232010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b590568-6a8f-462f-8dbf-b27ffaeec3a5 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:51.247612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df710d88-cd71-458f-9b62-acb096bcb0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:51.248341 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df710d88-cd71-458f-9b62-acb096bcb0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:51.248622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df710d88-cd71-458f-9b62-acb096bcb0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:51.248908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df710d88-cd71-458f-9b62-acb096bcb0e2 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:51.249644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df710d88-cd71-458f-9b62-acb096bcb0e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:51.251815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df710d88-cd71-458f-9b62-acb096bcb0e2 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:51.260482 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a56e9-cbcb-4eb6-af4a-61d2e035d261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:51.261178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a56e9-cbcb-4eb6-af4a-61d2e035d261 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:51.262211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a56e9-cbcb-4eb6-af4a-61d2e035d261 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:51.262496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a56e9-cbcb-4eb6-af4a-61d2e035d261 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:51.264348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a56e9-cbcb-4eb6-af4a-61d2e035d261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:51.266041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a56e9-cbcb-4eb6-af4a-61d2e035d261 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:51.280126 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 822/1623] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:51.281035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a56e9-cbcb-4eb6-af4a-61d2e035d261 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:51.292772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a56e9-cbcb-4eb6-af4a-61d2e035d261 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:51.298241 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5adc4cc-a53f-4fa1-a834-d34684af9f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:51.298988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5adc4cc-a53f-4fa1-a834-d34684af9f00 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:51.299944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5adc4cc-a53f-4fa1-a834-d34684af9f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:51.299944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5adc4cc-a53f-4fa1-a834-d34684af9f00 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:51.300500 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5adc4cc-a53f-4fa1-a834-d34684af9f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:51.302587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5adc4cc-a53f-4fa1-a834-d34684af9f00 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:51.347106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 802/1624] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:51 2022] POST /identity/v3/users => generated 296 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:51.347981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df710d88-cd71-458f-9b62-acb096bcb0e2 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:51.369595 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df710d88-cd71-458f-9b62-acb096bcb0e2 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:51.377430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62922a-3111-4aad-85ad-f01b50f1424e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:51.377430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62922a-3111-4aad-85ad-f01b50f1424e admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:51.377430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62922a-3111-4aad-85ad-f01b50f1424e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:51.377430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62922a-3111-4aad-85ad-f01b50f1424e admin admin] PATH_INFO: `/v3/projects/dc00b27d9949434189d02d2c078fa22f/users/0f7cca3c7f2e48608d5c31ccde963c39/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:51.379977 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62922a-3111-4aad-85ad-f01b50f1424e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc00b27d9949434189d02d2c078fa22f, user_id=0f7cca3c7f2e48608d5c31ccde963c39, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:51.383187 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62922a-3111-4aad-85ad-f01b50f1424e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:51.392047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:51.405648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 823/1625] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:51 2022] POST /identity/v3/users => generated 307 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:51.409905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5adc4cc-a53f-4fa1-a834-d34684af9f00 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:51.430280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 803/1626] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:51 2022] PUT /identity/v3/projects/dc00b27d9949434189d02d2c078fa22f/users/0f7cca3c7f2e48608d5c31ccde963c39/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:51.431474 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62922a-3111-4aad-85ad-f01b50f1424e admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:51.460488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5adc4cc-a53f-4fa1-a834-d34684af9f00 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:51.584211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62922a-3111-4aad-85ad-f01b50f1424e admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:51.605025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fc847-2679-414f-a003-ff3fd66ebd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:51.605025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fc847-2679-414f-a003-ff3fd66ebd69 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:51.605025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fc847-2679-414f-a003-ff3fd66ebd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:51.605025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fc847-2679-414f-a003-ff3fd66ebd69 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:51.605819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fc847-2679-414f-a003-ff3fd66ebd69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:51.605819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fc847-2679-414f-a003-ff3fd66ebd69 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:51.609280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d5355-565d-4914-a017-4b15bb607f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:51.610520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 804/1627] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:51.612485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d6b26-c41a-4849-a070-a731318ed02a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:51.612867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d6b26-c41a-4849-a070-a731318ed02a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:51.613202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d6b26-c41a-4849-a070-a731318ed02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:51.613953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d5355-565d-4914-a017-4b15bb607f3e admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:51.614359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d5355-565d-4914-a017-4b15bb607f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:51.614712 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d5355-565d-4914-a017-4b15bb607f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:51.615666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d5355-565d-4914-a017-4b15bb607f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:51.617659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2d5355-565d-4914-a017-4b15bb607f3e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:51.631963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 824/1628] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:51.633107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d5355-565d-4914-a017-4b15bb607f3e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:51.644167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f55d6b26-c41a-4849-a070-a731318ed02a None None] MFA Rules not processed for user `0f7cca3c7f2e48608d5c31ccde963c39`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:51.646547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f55d6b26-c41a-4849-a070-a731318ed02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:51.673097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d5355-565d-4914-a017-4b15bb607f3e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:51.686391 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb6901-a958-445b-b1bf-d35e7776cb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:51.689631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb6901-a958-445b-b1bf-d35e7776cb97 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:51.691624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb6901-a958-445b-b1bf-d35e7776cb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:51.692040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb6901-a958-445b-b1bf-d35e7776cb97 admin admin] PATH_INFO: `/v3/projects/f1b709ff31194d84b90f862a95a944bd/users/ed4e71d7251440eab9433947db0d2826/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:51.728326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb6901-a958-445b-b1bf-d35e7776cb97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b709ff31194d84b90f862a95a944bd, user_id=ed4e71d7251440eab9433947db0d2826, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:51.737815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb6901-a958-445b-b1bf-d35e7776cb97 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:51.783010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 805/1629] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:51 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:51.784366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d6b26-c41a-4849-a070-a731318ed02a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:51.797117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d6b26-c41a-4849-a070-a731318ed02a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:51.808432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d3772-7661-405c-9b07-4ca0a6a51f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:51.810461 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3772-7661-405c-9b07-4ca0a6a51f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:51.810988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3772-7661-405c-9b07-4ca0a6a51f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:51.813021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d3772-7661-405c-9b07-4ca0a6a51f47 admin admin] PATH_INFO: `/v3/projects/88fc6ce4c4e0474987ebedcfc10ce52a/users/ae97803f6ad14ddf8b689ca319bb28ff/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:51.835551 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d3772-7661-405c-9b07-4ca0a6a51f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88fc6ce4c4e0474987ebedcfc10ce52a, user_id=ae97803f6ad14ddf8b689ca319bb28ff, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:51.838803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d3772-7661-405c-9b07-4ca0a6a51f47 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:51.854693 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 825/1630] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:51 2022] PUT /identity/v3/projects/f1b709ff31194d84b90f862a95a944bd/users/ed4e71d7251440eab9433947db0d2826/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:51.854693 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb6901-a958-445b-b1bf-d35e7776cb97 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:51.975868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 806/1631] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:51 2022] PUT /identity/v3/projects/88fc6ce4c4e0474987ebedcfc10ce52a/users/ae97803f6ad14ddf8b689ca319bb28ff/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:51.978810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d3772-7661-405c-9b07-4ca0a6a51f47 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:51.982415 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb6901-a958-445b-b1bf-d35e7776cb97 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:52.092244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:52.125946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e69b81-9494-4b88-b51b-d0ae34555274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:52.128381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e69b81-9494-4b88-b51b-d0ae34555274 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:52.128892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e69b81-9494-4b88-b51b-d0ae34555274 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:52.129393 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e69b81-9494-4b88-b51b-d0ae34555274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:52.166967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d3772-7661-405c-9b07-4ca0a6a51f47 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:52.193671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4f9f7-a8cb-47f6-b37e-9abbf17e575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:52.194364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4f9f7-a8cb-47f6-b37e-9abbf17e575d admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:52.194644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4f9f7-a8cb-47f6-b37e-9abbf17e575d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:52.194918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4f9f7-a8cb-47f6-b37e-9abbf17e575d admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:52.196037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4f9f7-a8cb-47f6-b37e-9abbf17e575d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:52.197571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4f9f7-a8cb-47f6-b37e-9abbf17e575d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:52.225573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 807/1632] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:52.225573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4f9f7-a8cb-47f6-b37e-9abbf17e575d admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:52.251622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4f9f7-a8cb-47f6-b37e-9abbf17e575d admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:52.257181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15242c7-d081-4166-a310-6e11620a8c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:52.257875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15242c7-d081-4166-a310-6e11620a8c37 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:52.258156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15242c7-d081-4166-a310-6e11620a8c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:52.258430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15242c7-d081-4166-a310-6e11620a8c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:52.259148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15242c7-d081-4166-a310-6e11620a8c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:52.260775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15242c7-d081-4166-a310-6e11620a8c37 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:52.280401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e69b81-9494-4b88-b51b-d0ae34555274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:52.285291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 808/1633] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:52.285291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15242c7-d081-4166-a310-6e11620a8c37 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:52.285594 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e69b81-9494-4b88-b51b-d0ae34555274 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:52.300410 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 826/1634] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:51 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:55:52.301414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e69b81-9494-4b88-b51b-d0ae34555274 service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:52.313141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e69b81-9494-4b88-b51b-d0ae34555274 service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:52.313911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15242c7-d081-4166-a310-6e11620a8c37 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:52.320163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf94c6-9820-4f2c-b7a1-14bc6ed4c03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:52.321360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf94c6-9820-4f2c-b7a1-14bc6ed4c03e admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:52.321773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf94c6-9820-4f2c-b7a1-14bc6ed4c03e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:52.322114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf94c6-9820-4f2c-b7a1-14bc6ed4c03e admin admin] PATH_INFO: `/v3/projects/88fc6ce4c4e0474987ebedcfc10ce52a/users/ae97803f6ad14ddf8b689ca319bb28ff/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:52.324343 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf94c6-9820-4f2c-b7a1-14bc6ed4c03e admin admin] RBAC: Authorizing `identity:create_grant(project_id=88fc6ce4c4e0474987ebedcfc10ce52a, user_id=ae97803f6ad14ddf8b689ca319bb28ff, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:52.328685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf94c6-9820-4f2c-b7a1-14bc6ed4c03e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:52.339260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08ca76-89d7-4b1b-afe5-2457f9c3e050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:52.340178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08ca76-89d7-4b1b-afe5-2457f9c3e050 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:52.340456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08ca76-89d7-4b1b-afe5-2457f9c3e050 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:52.340734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08ca76-89d7-4b1b-afe5-2457f9c3e050 admin admin] PATH_INFO: `/v3/projects/f1b709ff31194d84b90f862a95a944bd/users/ed4e71d7251440eab9433947db0d2826/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:52.344639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08ca76-89d7-4b1b-afe5-2457f9c3e050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b709ff31194d84b90f862a95a944bd, user_id=ed4e71d7251440eab9433947db0d2826, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:52.348007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc08ca76-89d7-4b1b-afe5-2457f9c3e050 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:52.356613 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:52.367937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 827/1635] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:52 2022] PUT /identity/v3/projects/88fc6ce4c4e0474987ebedcfc10ce52a/users/ae97803f6ad14ddf8b689ca319bb28ff/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:52.371487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be81208-59b0-4811-a6f5-5952db5af9bc None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:52.372266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be81208-59b0-4811-a6f5-5952db5af9bc None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:52.374727 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be81208-59b0-4811-a6f5-5952db5af9bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:52.378771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 809/1636] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:52 2022] PUT /identity/v3/projects/f1b709ff31194d84b90f862a95a944bd/users/ed4e71d7251440eab9433947db0d2826/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:52.394987 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d355fc-6f71-46ef-8e9d-8d914ba42099 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:52.394987 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d355fc-6f71-46ef-8e9d-8d914ba42099 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:52.394987 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d355fc-6f71-46ef-8e9d-8d914ba42099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:52.428190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-2be81208-59b0-4811-a6f5-5952db5af9bc None None] MFA Rules not processed for user `ae97803f6ad14ddf8b689ca319bb28ff`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:52.430398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-2be81208-59b0-4811-a6f5-5952db5af9bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:52.445984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-44d355fc-6f71-46ef-8e9d-8d914ba42099 None None] MFA Rules not processed for user `ed4e71d7251440eab9433947db0d2826`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:52.458417 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-44d355fc-6f71-46ef-8e9d-8d914ba42099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:52.604784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 810/1637] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:52 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:52.618203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d355fc-6f71-46ef-8e9d-8d914ba42099 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:52.635265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 828/1638] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:52.758713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d355fc-6f71-46ef-8e9d-8d914ba42099 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:52.766838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be81208-59b0-4811-a6f5-5952db5af9bc None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:52.769915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb67ea9-f79b-4b9f-ab4e-585694f27835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:52.770702 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb67ea9-f79b-4b9f-ab4e-585694f27835 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:52.771840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb67ea9-f79b-4b9f-ab4e-585694f27835 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:52.771840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb67ea9-f79b-4b9f-ab4e-585694f27835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:52.784232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be81208-59b0-4811-a6f5-5952db5af9bc None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:52.794945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb67ea9-f79b-4b9f-ab4e-585694f27835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:52.795647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db542f-ad5f-4e83-9a2e-65ebf6b4b9c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:52.796553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db542f-ad5f-4e83-9a2e-65ebf6b4b9c9 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:52.796750 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb67ea9-f79b-4b9f-ab4e-585694f27835 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:52.797200 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db542f-ad5f-4e83-9a2e-65ebf6b4b9c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:52.799391 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db542f-ad5f-4e83-9a2e-65ebf6b4b9c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:52.822752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db542f-ad5f-4e83-9a2e-65ebf6b4b9c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:52.825613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53db542f-ad5f-4e83-9a2e-65ebf6b4b9c9 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:52.834190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 811/1639] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:52 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:55:52.849950 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 829/1640] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:55:52.928740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb67ea9-f79b-4b9f-ab4e-585694f27835 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:52.942089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb67ea9-f79b-4b9f-ab4e-585694f27835 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:52.951371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a514e8-ebb0-4855-b783-a019095cdecc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:52.952451 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a514e8-ebb0-4855-b783-a019095cdecc service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:52.952982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a514e8-ebb0-4855-b783-a019095cdecc service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:52.956175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a514e8-ebb0-4855-b783-a019095cdecc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:53.083121 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a514e8-ebb0-4855-b783-a019095cdecc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:53.090001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a514e8-ebb0-4855-b783-a019095cdecc service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:53.110989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 812/1641] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:55:53.387179 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db542f-ad5f-4e83-9a2e-65ebf6b4b9c9 service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:53.398704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db542f-ad5f-4e83-9a2e-65ebf6b4b9c9 service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:53.405722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0386c-9f24-4ddd-b6eb-51447b2e7c1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:53.406779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0386c-9f24-4ddd-b6eb-51447b2e7c1d service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:53.407331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0386c-9f24-4ddd-b6eb-51447b2e7c1d service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:53.407817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0386c-9f24-4ddd-b6eb-51447b2e7c1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:53.498828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:53.526466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0386c-9f24-4ddd-b6eb-51447b2e7c1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:53.528260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0386c-9f24-4ddd-b6eb-51447b2e7c1d service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:53.543411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 830/1642] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:55:53.700735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:53.797885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a514e8-ebb0-4855-b783-a019095cdecc service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:54.104889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780e616-3088-47e6-9d26-5c3567787f8c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:54.104889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780e616-3088-47e6-9d26-5c3567787f8c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:54.104889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780e616-3088-47e6-9d26-5c3567787f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:54.106164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a514e8-ebb0-4855-b783-a019095cdecc service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:54.107822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3780e616-3088-47e6-9d26-5c3567787f8c None None] MFA Rules not processed for user `4dc5d1e88e3a45588f671842e8cf9ffd`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:54.107822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3780e616-3088-47e6-9d26-5c3567787f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:54.108171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:54.108171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:54.108171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:54.108171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] PATH_INFO: `/v3/users/4ea14c9ded8e49cb8f6934d185831ca4` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:54.109731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ea14c9ded8e49cb8f6934d185831ca4)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:54.109731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: WARNING py.warnings [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:54.109731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: warnings.warn(msg) Jun 16 18:55:54.109731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: Jun 16 18:55:54.109731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:54.151892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 831/1643] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:53 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:54.156589 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780e616-3088-47e6-9d26-5c3567787f8c None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:54.192211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780e616-3088-47e6-9d26-5c3567787f8c None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:54.195849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fe207-5469-4304-bd4d-ce35cfff153b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:54.196546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fe207-5469-4304-bd4d-ce35cfff153b admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:54.196839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fe207-5469-4304-bd4d-ce35cfff153b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:54.197119 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fe207-5469-4304-bd4d-ce35cfff153b admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:54.200032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fe207-5469-4304-bd4d-ce35cfff153b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:54.204675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fe207-5469-4304-bd4d-ce35cfff153b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:54.424986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 832/1644] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:55:54 2022] POST /identity/v3/projects => generated 405 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:55:54.429307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fe207-5469-4304-bd4d-ce35cfff153b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:54.443927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ea14c9ded8e49cb8f6934d185831ca4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:54.479810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ea14c9ded8e49cb8f6934d185831ca4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:54.482292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ea14c9ded8e49cb8f6934d185831ca4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:54.525761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fe207-5469-4304-bd4d-ce35cfff153b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:54.531785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25232b8b-208d-4427-95de-31bcc9aba976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:54.532900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25232b8b-208d-4427-95de-31bcc9aba976 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:54.533290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25232b8b-208d-4427-95de-31bcc9aba976 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:54.533804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25232b8b-208d-4427-95de-31bcc9aba976 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:54.534491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ea14c9ded8e49cb8f6934d185831ca4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:54.534592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25232b8b-208d-4427-95de-31bcc9aba976 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:54.536245 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-cef4b9bc-6956-47d0-9b4e-efa8448542fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ea14c9ded8e49cb8f6934d185831ca4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:54.536596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25232b8b-208d-4427-95de-31bcc9aba976 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:54.672621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 813/1645] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:55:53 2022] DELETE /identity/v3/users/4ea14c9ded8e49cb8f6934d185831ca4 => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:54.678110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092668-b4e5-4577-94d7-aa27eca09d39 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:54.678654 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092668-b4e5-4577-94d7-aa27eca09d39 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:54.679149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16092668-b4e5-4577-94d7-aa27eca09d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:54.703595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 833/1646] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:55:54 2022] POST /identity/v3/users => generated 309 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:55:54.706815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25232b8b-208d-4427-95de-31bcc9aba976 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:54.709572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-16092668-b4e5-4577-94d7-aa27eca09d39 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:54.711089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-16092668-b4e5-4577-94d7-aa27eca09d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:54.711890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-25232b8b-208d-4427-95de-31bcc9aba976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:54.743164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25232b8b-208d-4427-95de-31bcc9aba976 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:54.841199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 814/1647] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:54.878750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030725be-280b-4092-b664-781cabe51e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:54.879458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030725be-280b-4092-b664-781cabe51e89 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:54.879745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030725be-280b-4092-b664-781cabe51e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:54.880417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030725be-280b-4092-b664-781cabe51e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:54.881299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030725be-280b-4092-b664-781cabe51e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:54.888907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030725be-280b-4092-b664-781cabe51e89 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:54.906196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 834/1648] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:55:54.907762 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16092668-b4e5-4577-94d7-aa27eca09d39 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:54.920450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16092668-b4e5-4577-94d7-aa27eca09d39 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:54.936260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fc1af-5015-4227-a67b-825a3c976718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:54.937203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fc1af-5015-4227-a67b-825a3c976718 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:54.937476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fc1af-5015-4227-a67b-825a3c976718 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:54.937753 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7fc1af-5015-4227-a67b-825a3c976718 admin admin] PATH_INFO: `/v3/projects/8c1d9c3add9349cd8f1c77b44f3d66f2/users/72f399fb172840d0bb04948a70a5f34b/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:54.940682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030725be-280b-4092-b664-781cabe51e89 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:54.942987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-030725be-280b-4092-b664-781cabe51e89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:54.978391 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fc1af-5015-4227-a67b-825a3c976718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c1d9c3add9349cd8f1c77b44f3d66f2, user_id=72f399fb172840d0bb04948a70a5f34b, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:54.981076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7fc1af-5015-4227-a67b-825a3c976718 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:55.037614 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:55.103122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 815/1649] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:54 2022] PUT /identity/v3/projects/8c1d9c3add9349cd8f1c77b44f3d66f2/users/72f399fb172840d0bb04948a70a5f34b/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:55.106861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fc1af-5015-4227-a67b-825a3c976718 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:55.114587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030725be-280b-4092-b664-781cabe51e89 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:55.231788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:55.233811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12073c5-25d2-4a87-be28-049b75869e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:55.234991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12073c5-25d2-4a87-be28-049b75869e6e service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:55.235282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12073c5-25d2-4a87-be28-049b75869e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:55.235561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12073c5-25d2-4a87-be28-049b75869e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:55.275951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7fc1af-5015-4227-a67b-825a3c976718 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:55.286434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a7d18-465a-4e22-8dfc-3c74b97499f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:55.293036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a7d18-465a-4e22-8dfc-3c74b97499f8 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:55.293358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a7d18-465a-4e22-8dfc-3c74b97499f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:55.293641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a7d18-465a-4e22-8dfc-3c74b97499f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:55.294415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a7d18-465a-4e22-8dfc-3c74b97499f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:55.303547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a7d18-465a-4e22-8dfc-3c74b97499f8 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:55.366728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 816/1650] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:55:55.374287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12073c5-25d2-4a87-be28-049b75869e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:55.378473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12073c5-25d2-4a87-be28-049b75869e6e service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:55.379054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a7d18-465a-4e22-8dfc-3c74b97499f8 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:55.405535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a7d18-465a-4e22-8dfc-3c74b97499f8 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:55.421094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86dc27-be0e-4c25-b4e3-6c296ecc4c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:55.423593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86dc27-be0e-4c25-b4e3-6c296ecc4c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:55.423593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86dc27-be0e-4c25-b4e3-6c296ecc4c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:55.423593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86dc27-be0e-4c25-b4e3-6c296ecc4c95 admin admin] PATH_INFO: `/v3/projects/8c1d9c3add9349cd8f1c77b44f3d66f2/users/72f399fb172840d0bb04948a70a5f34b/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:55.423928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 835/1651] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:55:55.426843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86dc27-be0e-4c25-b4e3-6c296ecc4c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c1d9c3add9349cd8f1c77b44f3d66f2, user_id=72f399fb172840d0bb04948a70a5f34b, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:55.440191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86dc27-be0e-4c25-b4e3-6c296ecc4c95 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:55.458971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 817/1652] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:55:55 2022] PUT /identity/v3/projects/8c1d9c3add9349cd8f1c77b44f3d66f2/users/72f399fb172840d0bb04948a70a5f34b/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:57.499746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12073c5-25d2-4a87-be28-049b75869e6e service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:57.500628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b12073c5-25d2-4a87-be28-049b75869e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:57.561983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12073c5-25d2-4a87-be28-049b75869e6e service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:57.646565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:57.669786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:57.671007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:57.671508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:57.671973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] PATH_INFO: `/v3/projects/dd6daa05f309410dbe5156c6e55fb175` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:57.680675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd6daa05f309410dbe5156c6e55fb175)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:57.680675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: WARNING py.warnings [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 18:55:57.680675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: warnings.warn(msg) Jun 16 18:55:57.680675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: Jun 16 18:55:57.692146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:57.927449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd6daa05f309410dbe5156c6e55fb175', 'request_id': 'req-71c10e84-6eea-4f24-80da-09c8a38aeec2'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:58.042212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd6daa05f309410dbe5156c6e55fb175', 'request_id': 'req-71c10e84-6eea-4f24-80da-09c8a38aeec2'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:58.058077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd6daa05f309410dbe5156c6e55fb175', 'request_id': 'req-71c10e84-6eea-4f24-80da-09c8a38aeec2'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:58.107960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] The token cache is being invalidate because project dd6daa05f309410dbe5156c6e55fb175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:55:58.108751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:58.109597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-71c10e84-6eea-4f24-80da-09c8a38aeec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:58.112380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 836/1653] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:55:57 2022] DELETE /identity/v3/projects/dd6daa05f309410dbe5156c6e55fb175 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:58.116316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86dc27-be0e-4c25-b4e3-6c296ecc4c95 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:58.119259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0e86dc27-be0e-4c25-b4e3-6c296ecc4c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:58.167944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86dc27-be0e-4c25-b4e3-6c296ecc4c95 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:58.205358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6250-fdb5-4aa0-af84-f0da7e58e0bd None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:58.205664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6250-fdb5-4aa0-af84-f0da7e58e0bd None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:58.205948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d6250-fdb5-4aa0-af84-f0da7e58e0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:58.278653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d61d6250-fdb5-4aa0-af84-f0da7e58e0bd None None] MFA Rules not processed for user `fec4e3ac2e44403b9d45b37b5a6f64a9`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:58.280014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d61d6250-fdb5-4aa0-af84-f0da7e58e0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:58.367153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:58.369159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:58.369616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:58.370056 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] PATH_INFO: `/v3/users/ed4e71d7251440eab9433947db0d2826` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:58.371708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4e71d7251440eab9433947db0d2826)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:58.384706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:58.476695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 837/1654] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:55:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:55:58.537596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b2d2f-6e15-4349-b073-5464c8f9505f None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:58.538161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b2d2f-6e15-4349-b073-5464c8f9505f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:58.538161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b2d2f-6e15-4349-b073-5464c8f9505f None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:58.540524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 838/1655] 10.222.0.81 () {54 vars in 961 bytes} [Thu Jun 16 18:55:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:55:58.572864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcf684-e4f3-4803-98d0-b13b42ed57c8 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:58.573519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcf684-e4f3-4803-98d0-b13b42ed57c8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:58.573971 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcf684-e4f3-4803-98d0-b13b42ed57c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:58.629939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-76dcf684-e4f3-4803-98d0-b13b42ed57c8 None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:55:58.631189 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-76dcf684-e4f3-4803-98d0-b13b42ed57c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:58.674842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4e71d7251440eab9433947db0d2826'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:58.696150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4e71d7251440eab9433947db0d2826'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:58.698153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4e71d7251440eab9433947db0d2826'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:58.743993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4e71d7251440eab9433947db0d2826'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:58.749978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4e71d7251440eab9433947db0d2826'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:55:58.761566 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:58.834698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:55:58.863466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 818/1656] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:55:58 2022] DELETE /identity/v3/users/ed4e71d7251440eab9433947db0d2826 => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:55:58.866704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 839/1657] 10.222.0.81 () {58 vars in 1058 bytes} [Thu Jun 16 18:55:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 18:55:58.868527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:55:59.072790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ca429-9dfc-4fc2-8018-b98aefe52fc9 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:55:59.080566 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774b455-8aa3-4544-a81b-6894a2422fa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:55:59.081585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774b455-8aa3-4544-a81b-6894a2422fa2 service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:55:59.085132 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774b455-8aa3-4544-a81b-6894a2422fa2 service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:55:59.085132 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774b455-8aa3-4544-a81b-6894a2422fa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:55:59.086449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8774b455-8aa3-4544-a81b-6894a2422fa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:55:59.174597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774b455-8aa3-4544-a81b-6894a2422fa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:55:59.176753 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774b455-8aa3-4544-a81b-6894a2422fa2 service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:55:59.188850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 819/1658] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:55:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:00.115580 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394a9e4-60f0-496c-bd94-c96c567c58b9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:00.116138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394a9e4-60f0-496c-bd94-c96c567c58b9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:00.116209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394a9e4-60f0-496c-bd94-c96c567c58b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:00.184420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-7394a9e4-60f0-496c-bd94-c96c567c58b9 None None] MFA Rules not processed for user `72f399fb172840d0bb04948a70a5f34b`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:00.191487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7394a9e4-60f0-496c-bd94-c96c567c58b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:00.319569 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774b455-8aa3-4544-a81b-6894a2422fa2 service nova] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:00.321082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8774b455-8aa3-4544-a81b-6894a2422fa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:00.390346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 840/1659] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:00 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:00.495578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774b455-8aa3-4544-a81b-6894a2422fa2 service nova] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:00.502981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:00.504081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:00.505307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:00.510833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] PATH_INFO: `/v3/projects/f1b709ff31194d84b90f862a95a944bd` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:00.510833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b709ff31194d84b90f862a95a944bd)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:00.512147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:00.638248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b709ff31194d84b90f862a95a944bd', 'request_id': 'req-70c342e5-feae-4fc6-9466-93ae24b70ce9'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:00.722803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b709ff31194d84b90f862a95a944bd', 'request_id': 'req-70c342e5-feae-4fc6-9466-93ae24b70ce9'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:00.731822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b709ff31194d84b90f862a95a944bd', 'request_id': 'req-70c342e5-feae-4fc6-9466-93ae24b70ce9'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:00.753343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] The token cache is being invalidate because project f1b709ff31194d84b90f862a95a944bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:00.754399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:00.755671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70c342e5-feae-4fc6-9466-93ae24b70ce9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:00.758876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 820/1660] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:00 2022] DELETE /identity/v3/projects/f1b709ff31194d84b90f862a95a944bd => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:00.763955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e957465-dfce-4095-ab9b-151620aac002 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:00.764254 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e957465-dfce-4095-ab9b-151620aac002 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:00.764542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e957465-dfce-4095-ab9b-151620aac002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:00.807290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-2e957465-dfce-4095-ab9b-151620aac002 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:00.811348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-2e957465-dfce-4095-ab9b-151620aac002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:00.985456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 841/1661] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:00.993514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345af7a4-da93-4775-a1c0-9bd876bee7c5 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:00.994047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345af7a4-da93-4775-a1c0-9bd876bee7c5 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:00.994589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345af7a4-da93-4775-a1c0-9bd876bee7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:01.030574 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-345af7a4-da93-4775-a1c0-9bd876bee7c5 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:01.032521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-345af7a4-da93-4775-a1c0-9bd876bee7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:01.050078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 821/1662] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:01.058630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e957465-dfce-4095-ab9b-151620aac002 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:01.071616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e957465-dfce-4095-ab9b-151620aac002 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:01.088392 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:01.090214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9797564-bda3-44db-ba57-b3d309705ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:01.091082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797564-bda3-44db-ba57-b3d309705ecb admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:01.092907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797564-bda3-44db-ba57-b3d309705ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:01.092907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797564-bda3-44db-ba57-b3d309705ecb admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:01.092907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9797564-bda3-44db-ba57-b3d309705ecb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:56:01.093222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9797564-bda3-44db-ba57-b3d309705ecb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:01.094890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9797564-bda3-44db-ba57-b3d309705ecb admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:01.123717 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 842/1663] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:56:01.125845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345af7a4-da93-4775-a1c0-9bd876bee7c5 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:01.142984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345af7a4-da93-4775-a1c0-9bd876bee7c5 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:01.149976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af35a84-097b-4266-bc3f-ef3fe78666fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:01.150899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af35a84-097b-4266-bc3f-ef3fe78666fd admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:01.151252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af35a84-097b-4266-bc3f-ef3fe78666fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:01.151593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af35a84-097b-4266-bc3f-ef3fe78666fd admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:01.152694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af35a84-097b-4266-bc3f-ef3fe78666fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:01.156770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af35a84-097b-4266-bc3f-ef3fe78666fd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:01.320535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 822/1664] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:01 2022] POST /identity/v3/projects => generated 371 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:01.324816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9797564-bda3-44db-ba57-b3d309705ecb admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:01.345706 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9797564-bda3-44db-ba57-b3d309705ecb admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:01.521105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a69c6a-68ee-44d8-b99f-6e45d57b3398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:01.522733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a69c6a-68ee-44d8-b99f-6e45d57b3398 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:01.523152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a69c6a-68ee-44d8-b99f-6e45d57b3398 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:01.523573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a69c6a-68ee-44d8-b99f-6e45d57b3398 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:01.524439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a69c6a-68ee-44d8-b99f-6e45d57b3398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:01.527203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a69c6a-68ee-44d8-b99f-6e45d57b3398 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:01.648980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:01.655282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 843/1665] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:01 2022] POST /identity/v3/users => generated 286 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:01.661932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af35a84-097b-4266-bc3f-ef3fe78666fd admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:01.688264 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af35a84-097b-4266-bc3f-ef3fe78666fd admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:01.714411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067f2a0-2601-4bcc-a14d-4cd372de43e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:01.714411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f2a0-2601-4bcc-a14d-4cd372de43e1 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:01.714411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f2a0-2601-4bcc-a14d-4cd372de43e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:01.714411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067f2a0-2601-4bcc-a14d-4cd372de43e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:01.714650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067f2a0-2601-4bcc-a14d-4cd372de43e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:01.719608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067f2a0-2601-4bcc-a14d-4cd372de43e1 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:01.736929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 823/1666] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:01.740805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a69c6a-68ee-44d8-b99f-6e45d57b3398 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:01.756784 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a69c6a-68ee-44d8-b99f-6e45d57b3398 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:01.778555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1469a5a0-b128-42c3-8c31-cba7cfd63e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:01.779375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469a5a0-b128-42c3-8c31-cba7cfd63e01 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:01.779997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469a5a0-b128-42c3-8c31-cba7cfd63e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:01.781340 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1469a5a0-b128-42c3-8c31-cba7cfd63e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:01.784086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469a5a0-b128-42c3-8c31-cba7cfd63e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:01.792536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1469a5a0-b128-42c3-8c31-cba7cfd63e01 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:01.861574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 844/1667] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:01.868235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067f2a0-2601-4bcc-a14d-4cd372de43e1 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:01.891394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067f2a0-2601-4bcc-a14d-4cd372de43e1 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:01.900025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ac8b7-8570-42ac-813a-7c4a6bb130bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:01.901059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ac8b7-8570-42ac-813a-7c4a6bb130bb admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:01.902667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ac8b7-8570-42ac-813a-7c4a6bb130bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:01.902667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ac8b7-8570-42ac-813a-7c4a6bb130bb admin admin] PATH_INFO: `/v3/projects/5f1be5bbf6de4b7d89bf438cd6a4d263/users/f5a460d9b61443009f0dce391904b7f5/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:01.924892 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ac8b7-8570-42ac-813a-7c4a6bb130bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f1be5bbf6de4b7d89bf438cd6a4d263, user_id=f5a460d9b61443009f0dce391904b7f5, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:01.928455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ac8b7-8570-42ac-813a-7c4a6bb130bb admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:02.033559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 824/1668] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:01 2022] PUT /identity/v3/projects/5f1be5bbf6de4b7d89bf438cd6a4d263/users/f5a460d9b61443009f0dce391904b7f5/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:02.033935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0b544-ecfd-4e6f-a052-c82902e46d1c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:02.034475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0b544-ecfd-4e6f-a052-c82902e46d1c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:02.034961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0b544-ecfd-4e6f-a052-c82902e46d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:02.089674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-43e0b544-ecfd-4e6f-a052-c82902e46d1c None None] MFA Rules not processed for user `f5a460d9b61443009f0dce391904b7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:02.091260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-43e0b544-ecfd-4e6f-a052-c82902e46d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:02.249580 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 845/1669] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:02.250417 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ac8b7-8570-42ac-813a-7c4a6bb130bb admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:02.402384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ac8b7-8570-42ac-813a-7c4a6bb130bb admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:02.414002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475792f-6f17-4bc4-9cd1-3fcbd592a6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:02.414711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475792f-6f17-4bc4-9cd1-3fcbd592a6ea admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:02.414999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475792f-6f17-4bc4-9cd1-3fcbd592a6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:02.415277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475792f-6f17-4bc4-9cd1-3fcbd592a6ea admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:02.415976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475792f-6f17-4bc4-9cd1-3fcbd592a6ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:02.425440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475792f-6f17-4bc4-9cd1-3fcbd592a6ea admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:02.530550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 825/1670] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:02 2022] POST /identity/v3/projects => generated 371 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:02.535894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0b544-ecfd-4e6f-a052-c82902e46d1c None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:02.556339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0b544-ecfd-4e6f-a052-c82902e46d1c None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:02.729177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437b0e6-d439-49b1-9a43-49ff92af0b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:02.730137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437b0e6-d439-49b1-9a43-49ff92af0b55 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:02.730590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437b0e6-d439-49b1-9a43-49ff92af0b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:02.731006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437b0e6-d439-49b1-9a43-49ff92af0b55 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:02.731841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437b0e6-d439-49b1-9a43-49ff92af0b55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:02.734655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437b0e6-d439-49b1-9a43-49ff92af0b55 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:02.842830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 846/1671] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:02 2022] POST /identity/v3/users => generated 292 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:02.847326 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475792f-6f17-4bc4-9cd1-3fcbd592a6ea admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:02.862345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475792f-6f17-4bc4-9cd1-3fcbd592a6ea admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:02.868210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ae263-60a4-4511-b91d-7286e513fc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:02.869042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ae263-60a4-4511-b91d-7286e513fc43 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:02.869368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ae263-60a4-4511-b91d-7286e513fc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:02.869654 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ae263-60a4-4511-b91d-7286e513fc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:02.874004 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ae263-60a4-4511-b91d-7286e513fc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:02.874004 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ae263-60a4-4511-b91d-7286e513fc43 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:02.921877 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 826/1672] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:02.925071 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437b0e6-d439-49b1-9a43-49ff92af0b55 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:02.944840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437b0e6-d439-49b1-9a43-49ff92af0b55 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:02.950079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906be20d-cadd-4bc9-aaba-13cea2e41ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:02.950744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906be20d-cadd-4bc9-aaba-13cea2e41ac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:02.950996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906be20d-cadd-4bc9-aaba-13cea2e41ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:02.951278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906be20d-cadd-4bc9-aaba-13cea2e41ac4 admin admin] PATH_INFO: `/v3/projects/d9ca02a8cde444b982cc18694fb00af6/users/63a80eaf89fe4fd7b741c18779674943/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:03.001643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906be20d-cadd-4bc9-aaba-13cea2e41ac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca02a8cde444b982cc18694fb00af6, user_id=63a80eaf89fe4fd7b741c18779674943, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:03.004188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906be20d-cadd-4bc9-aaba-13cea2e41ac4 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:03.099570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 847/1673] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:02 2022] PUT /identity/v3/projects/d9ca02a8cde444b982cc18694fb00af6/users/63a80eaf89fe4fd7b741c18779674943/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:03.103880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ae263-60a4-4511-b91d-7286e513fc43 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:03.120420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ae263-60a4-4511-b91d-7286e513fc43 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:03.168365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:03.241354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dcfd4-96d0-46ae-a32f-74c48f6b860e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:03.243868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dcfd4-96d0-46ae-a32f-74c48f6b860e admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:03.244391 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dcfd4-96d0-46ae-a32f-74c48f6b860e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:03.245822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dcfd4-96d0-46ae-a32f-74c48f6b860e admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:03.246851 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977dcfd4-96d0-46ae-a32f-74c48f6b860e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:03.253305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977dcfd4-96d0-46ae-a32f-74c48f6b860e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:03.284618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906be20d-cadd-4bc9-aaba-13cea2e41ac4 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:03.286494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 827/1674] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:03.324620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906be20d-cadd-4bc9-aaba-13cea2e41ac4 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:03.349828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62c5da-b472-43f4-8ff2-95773098ec9b None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:03.350285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62c5da-b472-43f4-8ff2-95773098ec9b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:03.350647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62c5da-b472-43f4-8ff2-95773098ec9b None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:03.352393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 828/1675] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 18:56:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:56:03.354676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3474f-07c7-4b8f-aa99-27eff4b02cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:03.355603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3474f-07c7-4b8f-aa99-27eff4b02cba admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:03.355884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3474f-07c7-4b8f-aa99-27eff4b02cba admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:03.356159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3474f-07c7-4b8f-aa99-27eff4b02cba admin admin] PATH_INFO: `/v3/projects/d9ca02a8cde444b982cc18694fb00af6/users/63a80eaf89fe4fd7b741c18779674943/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:03.373563 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3474f-07c7-4b8f-aa99-27eff4b02cba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ca02a8cde444b982cc18694fb00af6, user_id=63a80eaf89fe4fd7b741c18779674943, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:03.377144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3474f-07c7-4b8f-aa99-27eff4b02cba admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:03.430842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 848/1676] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:03 2022] PUT /identity/v3/projects/d9ca02a8cde444b982cc18694fb00af6/users/63a80eaf89fe4fd7b741c18779674943/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:03.433348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b060c64-46d0-4380-a247-74daaaf85310 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:03.435560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b060c64-46d0-4380-a247-74daaaf85310 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:03.436243 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b060c64-46d0-4380-a247-74daaaf85310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:03.469801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0727ad-d43c-43b2-bcc4-6e545da3af54 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:03.469801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0727ad-d43c-43b2-bcc4-6e545da3af54 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:03.470111 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0727ad-d43c-43b2-bcc4-6e545da3af54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:03.477152 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6b060c64-46d0-4380-a247-74daaaf85310 None None] MFA Rules not processed for user `63a80eaf89fe4fd7b741c18779674943`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:03.478918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6b060c64-46d0-4380-a247-74daaaf85310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:03.527458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ef0727ad-d43c-43b2-bcc4-6e545da3af54 None None] MFA Rules not processed for user `20c0fd976e684356b6143dc33fb7b4af`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:03.528859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ef0727ad-d43c-43b2-bcc4-6e545da3af54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:03.601658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 829/1677] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:03.609692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:03.615920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b060c64-46d0-4380-a247-74daaaf85310 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:03.618466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6b060c64-46d0-4380-a247-74daaaf85310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:03.672519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 849/1678] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 18:56:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 18:56:03.703670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7494f-9200-4b7a-83b2-0d8af31f339d None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:03.703962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7494f-9200-4b7a-83b2-0d8af31f339d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:03.704280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7494f-9200-4b7a-83b2-0d8af31f339d None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:03.707467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 850/1679] 10.222.0.81 () {54 vars in 960 bytes} [Thu Jun 16 18:56:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:56:03.713550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59248a58-661b-47cb-820c-a42a5f84bbbd None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:03.713818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59248a58-661b-47cb-820c-a42a5f84bbbd None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:03.714137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59248a58-661b-47cb-820c-a42a5f84bbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:03.759800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-59248a58-661b-47cb-820c-a42a5f84bbbd None None] MFA Rules not processed for user `20c0fd976e684356b6143dc33fb7b4af`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:03.764645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-59248a58-661b-47cb-820c-a42a5f84bbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:03.776840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b060c64-46d0-4380-a247-74daaaf85310 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:03.788709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c275e3-68e8-44ea-b25c-a9f9ad70d158 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:03.789426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c275e3-68e8-44ea-b25c-a9f9ad70d158 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:03.789717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c275e3-68e8-44ea-b25c-a9f9ad70d158 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:03.789997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c275e3-68e8-44ea-b25c-a9f9ad70d158 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:03.791811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 851/1680] 10.222.0.81 () {58 vars in 1057 bytes} [Thu Jun 16 18:56:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:03.802108 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59248a58-661b-47cb-820c-a42a5f84bbbd None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:03.815575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c275e3-68e8-44ea-b25c-a9f9ad70d158 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:03.817537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c275e3-68e8-44ea-b25c-a9f9ad70d158 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:03.818426 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59248a58-661b-47cb-820c-a42a5f84bbbd None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:03.835665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be431031-a861-464f-830e-3a23a2f9a321 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:03.836333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be431031-a861-464f-830e-3a23a2f9a321 service placement] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:03.836616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be431031-a861-464f-830e-3a23a2f9a321 service placement] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:03.836890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be431031-a861-464f-830e-3a23a2f9a321 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:03.855383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be431031-a861-464f-830e-3a23a2f9a321 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:03.855773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 830/1681] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:03 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:03.856993 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be431031-a861-464f-830e-3a23a2f9a321 service placement] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:03.876713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 852/1682] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:04.186736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c275e3-68e8-44ea-b25c-a9f9ad70d158 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:04.195779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c275e3-68e8-44ea-b25c-a9f9ad70d158 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:04.212913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c378b-0774-4fb5-9b0c-2c38a4cf83a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:04.214198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c378b-0774-4fb5-9b0c-2c38a4cf83a1 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:04.214198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c378b-0774-4fb5-9b0c-2c38a4cf83a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:04.214198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c378b-0774-4fb5-9b0c-2c38a4cf83a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:04.346397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c378b-0774-4fb5-9b0c-2c38a4cf83a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:04.346397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c378b-0774-4fb5-9b0c-2c38a4cf83a1 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:04.380367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 831/1683] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:04.807781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:05.309688 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c48910-2569-4e20-9768-a06aa123a713 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:05.310164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c48910-2569-4e20-9768-a06aa123a713 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:05.310263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c48910-2569-4e20-9768-a06aa123a713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:05.368767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-25c48910-2569-4e20-9768-a06aa123a713 None None] MFA Rules not processed for user `1052dda92d1a4597be43a7acdcd06905`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:05.370534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-25c48910-2569-4e20-9768-a06aa123a713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:05.523049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 853/1684] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:05.534026 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb399e-052d-4f4d-9ef6-447f8eef98cc None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:05.535079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb399e-052d-4f4d-9ef6-447f8eef98cc None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:05.535502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb399e-052d-4f4d-9ef6-447f8eef98cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:05.600656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6ddb399e-052d-4f4d-9ef6-447f8eef98cc None None] MFA Rules not processed for user `1052dda92d1a4597be43a7acdcd06905`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:05.616150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6ddb399e-052d-4f4d-9ef6-447f8eef98cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:05.650786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 832/1685] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:05.705357 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e08f1-02da-427e-a0a6-dc8f71565544 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:05.705660 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e08f1-02da-427e-a0a6-dc8f71565544 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:05.705938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e08f1-02da-427e-a0a6-dc8f71565544 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:05.710698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 854/1686] 10.222.0.81 () {54 vars in 961 bytes} [Thu Jun 16 18:56:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:56:05.744125 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa52e5-5287-4b1a-9332-f5dffa473fb6 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:05.744125 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa52e5-5287-4b1a-9332-f5dffa473fb6 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:05.744125 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa52e5-5287-4b1a-9332-f5dffa473fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:05.779561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-61aa52e5-5287-4b1a-9332-f5dffa473fb6 None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:05.781719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-61aa52e5-5287-4b1a-9332-f5dffa473fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:05.804626 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e08f1-02da-427e-a0a6-dc8f71565544 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:05.806739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f04e08f1-02da-427e-a0a6-dc8f71565544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:05.933281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e08f1-02da-427e-a0a6-dc8f71565544 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:05.994469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e04b76-3e23-4b14-9685-56a89d2247e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:05.994469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e04b76-3e23-4b14-9685-56a89d2247e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:05.994469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e04b76-3e23-4b14-9685-56a89d2247e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:05.994469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e04b76-3e23-4b14-9685-56a89d2247e7 admin admin] PATH_INFO: `/v3/users/4dc5d1e88e3a45588f671842e8cf9ffd` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:05.994998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e04b76-3e23-4b14-9685-56a89d2247e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dc5d1e88e3a45588f671842e8cf9ffd)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:06.006942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e04b76-3e23-4b14-9685-56a89d2247e7 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:06.064793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 833/1687] 10.222.0.81 () {58 vars in 1058 bytes} [Thu Jun 16 18:56:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:06.065845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa52e5-5287-4b1a-9332-f5dffa473fb6 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:06.115509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa52e5-5287-4b1a-9332-f5dffa473fb6 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:06.127474 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c89d9-d042-4537-b49c-e7686165ef73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:06.128221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c89d9-d042-4537-b49c-e7686165ef73 service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:06.128816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c89d9-d042-4537-b49c-e7686165ef73 service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:06.129102 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c89d9-d042-4537-b49c-e7686165ef73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:06.156950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c89d9-d042-4537-b49c-e7686165ef73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:06.159673 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c89d9-d042-4537-b49c-e7686165ef73 service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:06.201376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 834/1688] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:06.381226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:06.394823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-64e04b76-3e23-4b14-9685-56a89d2247e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dc5d1e88e3a45588f671842e8cf9ffd'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:06.396772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-64e04b76-3e23-4b14-9685-56a89d2247e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dc5d1e88e3a45588f671842e8cf9ffd'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:06.412346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-64e04b76-3e23-4b14-9685-56a89d2247e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dc5d1e88e3a45588f671842e8cf9ffd'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:06.414615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-64e04b76-3e23-4b14-9685-56a89d2247e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dc5d1e88e3a45588f671842e8cf9ffd'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:06.465173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-64e04b76-3e23-4b14-9685-56a89d2247e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dc5d1e88e3a45588f671842e8cf9ffd'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:06.537730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bcc6d-df61-415f-8ed2-2f6e5b85ae6b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:06.538035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bcc6d-df61-415f-8ed2-2f6e5b85ae6b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:06.538343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bcc6d-df61-415f-8ed2-2f6e5b85ae6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:06.595215 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-079bcc6d-df61-415f-8ed2-2f6e5b85ae6b None None] MFA Rules not processed for user `7310c631282c495a83e17dbdbc6d2ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:06.596727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-079bcc6d-df61-415f-8ed2-2f6e5b85ae6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:06.608682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 855/1689] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:05 2022] DELETE /identity/v3/users/4dc5d1e88e3a45588f671842e8cf9ffd => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:56:06.703026 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 835/1690] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:06.715159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e92e71-6283-497b-9767-06df8f73f56e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:06.715766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e92e71-6283-497b-9767-06df8f73f56e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:06.716186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e92e71-6283-497b-9767-06df8f73f56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:06.743577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a5f12-4788-484a-be2a-2dde0285fa54 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:06.744238 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a5f12-4788-484a-be2a-2dde0285fa54 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:06.744723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a5f12-4788-484a-be2a-2dde0285fa54 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:06.747150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 836/1691] 10.222.0.81 () {54 vars in 912 bytes} [Thu Jun 16 18:56:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:56:06.754018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c23ee1-b191-423d-921e-cb5a4e8d60d3 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:06.754661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c23ee1-b191-423d-921e-cb5a4e8d60d3 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:06.755144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c23ee1-b191-423d-921e-cb5a4e8d60d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:06.764962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f2e92e71-6283-497b-9767-06df8f73f56e None None] MFA Rules not processed for user `7310c631282c495a83e17dbdbc6d2ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:06.769822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f2e92e71-6283-497b-9767-06df8f73f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:06.792373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-52c23ee1-b191-423d-921e-cb5a4e8d60d3 None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:06.794282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-52c23ee1-b191-423d-921e-cb5a4e8d60d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:06.867341 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:06.912397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 837/1692] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 18:56:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:06.914535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c23ee1-b191-423d-921e-cb5a4e8d60d3 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:06.916975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-52c23ee1-b191-423d-921e-cb5a4e8d60d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:06.940464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 856/1693] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:06.953221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e92e71-6283-497b-9767-06df8f73f56e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:06.955003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f2e92e71-6283-497b-9767-06df8f73f56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:07.001481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e92e71-6283-497b-9767-06df8f73f56e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:07.008647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c23ee1-b191-423d-921e-cb5a4e8d60d3 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:07.013889 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58938f1-e879-4d8e-aa46-cbf4a72ef786 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17b7a7128d64b2ba548add02d3ec264', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:07.014823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58938f1-e879-4d8e-aa46-cbf4a72ef786 service glance] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:07.015430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58938f1-e879-4d8e-aa46-cbf4a72ef786 service glance] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:07.016388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58938f1-e879-4d8e-aa46-cbf4a72ef786 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:07.028100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c0b4d-5c2d-4645-a01f-8096c336791b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:07.028801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c0b4d-5c2d-4645-a01f-8096c336791b service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:07.029090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c0b4d-5c2d-4645-a01f-8096c336791b service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:07.029402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c0b4d-5c2d-4645-a01f-8096c336791b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:07.051284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58938f1-e879-4d8e-aa46-cbf4a72ef786 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:07.055242 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58938f1-e879-4d8e-aa46-cbf4a72ef786 service glance] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:07.065812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c0b4d-5c2d-4645-a01f-8096c336791b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:07.067435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c0b4d-5c2d-4645-a01f-8096c336791b service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:07.071507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 838/1694] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:07.073115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7873fa-1315-4ac9-a9d3-4d43f3d8bc7c None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:07.073441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7873fa-1315-4ac9-a9d3-4d43f3d8bc7c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:07.074160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7873fa-1315-4ac9-a9d3-4d43f3d8bc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:07.096094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 857/1695] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:56:07.100197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-5b7873fa-1315-4ac9-a9d3-4d43f3d8bc7c None None] MFA Rules not processed for user `fec4e3ac2e44403b9d45b37b5a6f64a9`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:07.102176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5b7873fa-1315-4ac9-a9d3-4d43f3d8bc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:07.106194 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c0b4d-5c2d-4645-a01f-8096c336791b service nova] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:07.126330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-376c0b4d-5c2d-4645-a01f-8096c336791b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:07.237956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 839/1696] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:07.239594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d33842-629e-4732-b385-58b61f5f35e1 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:07.241188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d33842-629e-4732-b385-58b61f5f35e1 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:07.241925 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d33842-629e-4732-b385-58b61f5f35e1 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:07.246165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 840/1697] 10.222.0.81 () {54 vars in 912 bytes} [Thu Jun 16 18:56:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:56:07.250680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422fda5-f8af-4e8b-b9f8-315d4a9d9a91 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:07.251605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422fda5-f8af-4e8b-b9f8-315d4a9d9a91 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:07.252050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422fda5-f8af-4e8b-b9f8-315d4a9d9a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:07.293961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2422fda5-f8af-4e8b-b9f8-315d4a9d9a91 None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:07.296603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2422fda5-f8af-4e8b-b9f8-315d4a9d9a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:07.320613 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 841/1698] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 18:56:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:07.332662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c0b4d-5c2d-4645-a01f-8096c336791b service nova] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:07.339918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422fda5-f8af-4e8b-b9f8-315d4a9d9a91 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:07.340795 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2422fda5-f8af-4e8b-b9f8-315d4a9d9a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:07.344749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:07.345668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:07.345955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:07.346231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] PATH_INFO: `/v3/projects/061a027ac5464038b15bbca6c47e7f7c` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:07.349278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=061a027ac5464038b15bbca6c47e7f7c)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:07.349644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:07.362725 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422fda5-f8af-4e8b-b9f8-315d4a9d9a91 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:07.370074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee4b13-857e-4d2b-911e-17c55a656922 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17b7a7128d64b2ba548add02d3ec264', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:07.370946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee4b13-857e-4d2b-911e-17c55a656922 service glance] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:07.371298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee4b13-857e-4d2b-911e-17c55a656922 service glance] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:07.371656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee4b13-857e-4d2b-911e-17c55a656922 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:07.402370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee4b13-857e-4d2b-911e-17c55a656922 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:07.404484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee4b13-857e-4d2b-911e-17c55a656922 service glance] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:07.435733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 842/1699] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:07.574235 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061a027ac5464038b15bbca6c47e7f7c', 'request_id': 'req-ea005400-3d4e-4393-b71e-b90887974bbf'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:07.727108 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061a027ac5464038b15bbca6c47e7f7c', 'request_id': 'req-ea005400-3d4e-4393-b71e-b90887974bbf'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:07.734839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061a027ac5464038b15bbca6c47e7f7c', 'request_id': 'req-ea005400-3d4e-4393-b71e-b90887974bbf'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:07.763923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] The token cache is being invalidate because project 061a027ac5464038b15bbca6c47e7f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:07.764431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:07.765284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ea005400-3d4e-4393-b71e-b90887974bbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:07.782275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 858/1700] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:07 2022] DELETE /identity/v3/projects/061a027ac5464038b15bbca6c47e7f7c => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:07.789108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee4b13-857e-4d2b-911e-17c55a656922 service glance] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:07.801360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee4b13-857e-4d2b-911e-17c55a656922 service glance] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:07.890553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7908cead-9cc7-4535-ab91-f0eb32373fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:07.891857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908cead-9cc7-4535-ab91-f0eb32373fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:07.892706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908cead-9cc7-4535-ab91-f0eb32373fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:07.892998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7908cead-9cc7-4535-ab91-f0eb32373fc1 admin admin] PATH_INFO: `/v3/users/72f399fb172840d0bb04948a70a5f34b` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:07.900064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908cead-9cc7-4535-ab91-f0eb32373fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f399fb172840d0bb04948a70a5f34b)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:07.903158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7908cead-9cc7-4535-ab91-f0eb32373fc1 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:07.918276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaf5c3-99e1-471a-a0e9-b7ce4e0e3b6f None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:07.918695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaf5c3-99e1-471a-a0e9-b7ce4e0e3b6f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:07.922977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaf5c3-99e1-471a-a0e9-b7ce4e0e3b6f None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:07.925418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 859/1701] 10.222.0.81 () {54 vars in 912 bytes} [Thu Jun 16 18:56:07 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:56:07.951419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790c858-67d5-4e81-9bf9-982b3de544ba None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:07.951708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790c858-67d5-4e81-9bf9-982b3de544ba None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:07.951989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790c858-67d5-4e81-9bf9-982b3de544ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:07.954229 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:08.065442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0790c858-67d5-4e81-9bf9-982b3de544ba None None] MFA Rules not processed for user `f4e05d75cd1142e18d7518b78edd88fa`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:08.066779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0790c858-67d5-4e81-9bf9-982b3de544ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:08.154439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7908cead-9cc7-4535-ab91-f0eb32373fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f399fb172840d0bb04948a70a5f34b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:08.162223 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7908cead-9cc7-4535-ab91-f0eb32373fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f399fb172840d0bb04948a70a5f34b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:08.163089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7908cead-9cc7-4535-ab91-f0eb32373fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f399fb172840d0bb04948a70a5f34b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:08.171239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 860/1702] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 18:56:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:08.171239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1850fe-5e4a-4371-aecd-f662bf62576b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:08.171239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1850fe-5e4a-4371-aecd-f662bf62576b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:08.171239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1850fe-5e4a-4371-aecd-f662bf62576b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:08.185463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7908cead-9cc7-4535-ab91-f0eb32373fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f399fb172840d0bb04948a70a5f34b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:08.186222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7908cead-9cc7-4535-ab91-f0eb32373fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f399fb172840d0bb04948a70a5f34b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:08.229054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ba1850fe-5e4a-4371-aecd-f662bf62576b None None] MFA Rules not processed for user `2de523abbc7b4f8b972dc97b8a013237`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:08.229054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ba1850fe-5e4a-4371-aecd-f662bf62576b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:08.304138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 861/1703] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:08.305146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748c2a5-a77d-4e94-94f7-dabfd1d72a19 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:08.305549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748c2a5-a77d-4e94-94f7-dabfd1d72a19 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:08.305836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748c2a5-a77d-4e94-94f7-dabfd1d72a19 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:08.307504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 862/1704] 10.222.0.81 () {54 vars in 912 bytes} [Thu Jun 16 18:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:56:08.308083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748c2a5-a77d-4e94-94f7-dabfd1d72a19 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:08.312061 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 843/1705] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:07 2022] DELETE /identity/v3/users/72f399fb172840d0bb04948a70a5f34b => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:08.317857 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8748c2a5-a77d-4e94-94f7-dabfd1d72a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:08.323784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe19b8b-55dd-4069-b799-b4c01b1fd5a4 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:08.324078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe19b8b-55dd-4069-b799-b4c01b1fd5a4 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:08.324365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe19b8b-55dd-4069-b799-b4c01b1fd5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:08.396387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-afe19b8b-55dd-4069-b799-b4c01b1fd5a4 None None] MFA Rules not processed for user `f4e05d75cd1142e18d7518b78edd88fa`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:08.398052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-afe19b8b-55dd-4069-b799-b4c01b1fd5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:08.417228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:08.445278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748c2a5-a77d-4e94-94f7-dabfd1d72a19 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:08.450248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9d6c9-06be-46af-baa6-d2d49b03d4eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:08.452786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9d6c9-06be-46af-baa6-d2d49b03d4eb service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:08.453114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9d6c9-06be-46af-baa6-d2d49b03d4eb service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:08.453426 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9d6c9-06be-46af-baa6-d2d49b03d4eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:08.464913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9d6c9-06be-46af-baa6-d2d49b03d4eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:08.471629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9d6c9-06be-46af-baa6-d2d49b03d4eb service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:08.483824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 863/1706] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:08.505587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 844/1707] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 18:56:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:08.545464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9d6c9-06be-46af-baa6-d2d49b03d4eb service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:08.561345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9d6c9-06be-46af-baa6-d2d49b03d4eb service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:08.567068 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90727974-c3f1-4ba6-9a4a-b064d14c5ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:08.568196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90727974-c3f1-4ba6-9a4a-b064d14c5ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:08.568673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90727974-c3f1-4ba6-9a4a-b064d14c5ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:08.569158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90727974-c3f1-4ba6-9a4a-b064d14c5ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:08.583000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90727974-c3f1-4ba6-9a4a-b064d14c5ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:08.585399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90727974-c3f1-4ba6-9a4a-b064d14c5ac1 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:08.601733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 864/1708] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:08.970014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe19b8b-55dd-4069-b799-b4c01b1fd5a4 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:08.971964 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-afe19b8b-55dd-4069-b799-b4c01b1fd5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:09.279602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe19b8b-55dd-4069-b799-b4c01b1fd5a4 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:09.305002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:09.309097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:09.309097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:09.309097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] PATH_INFO: `/v3/projects/8c1d9c3add9349cd8f1c77b44f3d66f2` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:09.310668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c1d9c3add9349cd8f1c77b44f3d66f2)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:09.312864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:09.507931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c1d9c3add9349cd8f1c77b44f3d66f2', 'request_id': 'req-74649d95-0b4b-4ca3-9cfa-f26b20034a34'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:09.576436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90727974-c3f1-4ba6-9a4a-b064d14c5ac1 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:09.578239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-90727974-c3f1-4ba6-9a4a-b064d14c5ac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:09.584298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c1d9c3add9349cd8f1c77b44f3d66f2', 'request_id': 'req-74649d95-0b4b-4ca3-9cfa-f26b20034a34'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:09.592321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c1d9c3add9349cd8f1c77b44f3d66f2', 'request_id': 'req-74649d95-0b4b-4ca3-9cfa-f26b20034a34'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:09.608127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] The token cache is being invalidate because project 8c1d9c3add9349cd8f1c77b44f3d66f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:09.608427 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:09.609985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-74649d95-0b4b-4ca3-9cfa-f26b20034a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:09.616270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 845/1709] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:08 2022] DELETE /identity/v3/projects/8c1d9c3add9349cd8f1c77b44f3d66f2 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:09.618472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fed324-6fc2-4661-b9d4-387a528874fd None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:09.618774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fed324-6fc2-4661-b9d4-387a528874fd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:09.619070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fed324-6fc2-4661-b9d4-387a528874fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:09.687447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d5fed324-6fc2-4661-b9d4-387a528874fd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:09.690165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d5fed324-6fc2-4661-b9d4-387a528874fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:09.733575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:09.848798 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90727974-c3f1-4ba6-9a4a-b064d14c5ac1 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:09.930260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 846/1710] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:09.931202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fed324-6fc2-4661-b9d4-387a528874fd None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:09.934335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d5fed324-6fc2-4661-b9d4-387a528874fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:10.016834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcf444-773b-4c6b-913e-57a208248af2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:10.017643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf444-773b-4c6b-913e-57a208248af2 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:10.018199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf444-773b-4c6b-913e-57a208248af2 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:10.018755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcf444-773b-4c6b-913e-57a208248af2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:10.022627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-35fcf444-773b-4c6b-913e-57a208248af2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:10.066170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fed324-6fc2-4661-b9d4-387a528874fd None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:10.086208 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d208176-f4e5-4431-9402-27bf6847333a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:10.086892 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208176-f4e5-4431-9402-27bf6847333a service placement] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:10.087182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208176-f4e5-4431-9402-27bf6847333a service placement] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:10.087468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d208176-f4e5-4431-9402-27bf6847333a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:10.094402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6d208176-f4e5-4431-9402-27bf6847333a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:10.104104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcf444-773b-4c6b-913e-57a208248af2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:10.106287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcf444-773b-4c6b-913e-57a208248af2 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:10.120504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d208176-f4e5-4431-9402-27bf6847333a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:10.122189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d208176-f4e5-4431-9402-27bf6847333a service placement] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:10.136694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 865/1711] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:56:10.139676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcf444-773b-4c6b-913e-57a208248af2 service nova] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:10.142279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-35fcf444-773b-4c6b-913e-57a208248af2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:10.154637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 847/1712] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:10.156113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4bc0a-c917-4474-a92c-d73c29dfa4a0 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:10.156407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4bc0a-c917-4474-a92c-d73c29dfa4a0 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:10.162292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4bc0a-c917-4474-a92c-d73c29dfa4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:10.170183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcf444-773b-4c6b-913e-57a208248af2 service nova] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:10.184983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4612c2-bf12-4574-9095-f15c20a096d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:10.185743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4612c2-bf12-4574-9095-f15c20a096d6 service placement] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:10.186034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4612c2-bf12-4574-9095-f15c20a096d6 service placement] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:10.186316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4612c2-bf12-4574-9095-f15c20a096d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:10.206476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-60a4bc0a-c917-4474-a92c-d73c29dfa4a0 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:10.208059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-60a4bc0a-c917-4474-a92c-d73c29dfa4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:10.209745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4612c2-bf12-4574-9095-f15c20a096d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:10.211770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4612c2-bf12-4574-9095-f15c20a096d6 service placement] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:10.236930 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 848/1713] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:10.244169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4bc0a-c917-4474-a92c-d73c29dfa4a0 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:10.248729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 866/1714] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:10.264262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4bc0a-c917-4474-a92c-d73c29dfa4a0 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:10.270553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f070b-7041-4e09-8883-6ac413dfe152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:10.271957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f070b-7041-4e09-8883-6ac413dfe152 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:10.272391 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f070b-7041-4e09-8883-6ac413dfe152 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:10.272812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f070b-7041-4e09-8883-6ac413dfe152 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:10.274074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f070b-7041-4e09-8883-6ac413dfe152 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:56:10.274559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f070b-7041-4e09-8883-6ac413dfe152 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:10.310694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f070b-7041-4e09-8883-6ac413dfe152 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:10.346276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 849/1715] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:56:10.350411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4612c2-bf12-4574-9095-f15c20a096d6 service placement] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:10.365467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4612c2-bf12-4574-9095-f15c20a096d6 service placement] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:10.371307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc517-9fba-48a9-a3b4-e47fcdf50129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:10.372075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc517-9fba-48a9-a3b4-e47fcdf50129 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:10.372432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc517-9fba-48a9-a3b4-e47fcdf50129 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:10.372779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cc517-9fba-48a9-a3b4-e47fcdf50129 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:10.373684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cc517-9fba-48a9-a3b4-e47fcdf50129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:10.376212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3cc517-9fba-48a9-a3b4-e47fcdf50129 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:10.497413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 867/1716] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:10 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:10.502717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f070b-7041-4e09-8883-6ac413dfe152 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:10.516165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f070b-7041-4e09-8883-6ac413dfe152 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:10.545173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:10.753667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48694a8a-99f4-4f5a-a904-fd2050b0fbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:10.754163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48694a8a-99f4-4f5a-a904-fd2050b0fbf8 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:10.754452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48694a8a-99f4-4f5a-a904-fd2050b0fbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:10.754732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48694a8a-99f4-4f5a-a904-fd2050b0fbf8 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:10.755465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48694a8a-99f4-4f5a-a904-fd2050b0fbf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:10.764059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48694a8a-99f4-4f5a-a904-fd2050b0fbf8 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:10.913476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 850/1717] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:10 2022] POST /identity/v3/users => generated 285 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:10.915992 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc517-9fba-48a9-a3b4-e47fcdf50129 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:10.941381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3cc517-9fba-48a9-a3b4-e47fcdf50129 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:10.965879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee52179-1533-4500-98d3-4f6bae875a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:10.966919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee52179-1533-4500-98d3-4f6bae875a95 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:10.967216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee52179-1533-4500-98d3-4f6bae875a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:10.967509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee52179-1533-4500-98d3-4f6bae875a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:10.972243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee52179-1533-4500-98d3-4f6bae875a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:10.972243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee52179-1533-4500-98d3-4f6bae875a95 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:11.008947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea689f-7d6f-4ca3-8ac0-211a59d478ea None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:11.008947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea689f-7d6f-4ca3-8ac0-211a59d478ea None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:11.008947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea689f-7d6f-4ca3-8ac0-211a59d478ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:11.044083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 868/1718] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:11.045139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee52179-1533-4500-98d3-4f6bae875a95 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:11.074466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-86ea689f-7d6f-4ca3-8ac0-211a59d478ea None None] MFA Rules not processed for user `7310c631282c495a83e17dbdbc6d2ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:11.078822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee52179-1533-4500-98d3-4f6bae875a95 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:11.079880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-86ea689f-7d6f-4ca3-8ac0-211a59d478ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:11.098492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3814d-2549-4fae-9814-85ecdb418895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:11.099259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3814d-2549-4fae-9814-85ecdb418895 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:11.099552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3814d-2549-4fae-9814-85ecdb418895 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:11.099763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3814d-2549-4fae-9814-85ecdb418895 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:11.100494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3814d-2549-4fae-9814-85ecdb418895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:11.109304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3814d-2549-4fae-9814-85ecdb418895 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:11.136146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 869/1719] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:11.137064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3814d-2549-4fae-9814-85ecdb418895 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:11.170845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3814d-2549-4fae-9814-85ecdb418895 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:11.185621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f99ff-0ccb-4400-abe9-fdc21ef8cdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:11.189181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f99ff-0ccb-4400-abe9-fdc21ef8cdff admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:11.192658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f99ff-0ccb-4400-abe9-fdc21ef8cdff admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:11.193102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f99ff-0ccb-4400-abe9-fdc21ef8cdff admin admin] PATH_INFO: `/v3/projects/9aacf82a037e4759a811b9c064510932/users/e0b39cbbaa4049af8d89e9a400b00abb/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:11.229954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f99ff-0ccb-4400-abe9-fdc21ef8cdff admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aacf82a037e4759a811b9c064510932, user_id=e0b39cbbaa4049af8d89e9a400b00abb, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:11.238531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f99ff-0ccb-4400-abe9-fdc21ef8cdff admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:11.243891 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 851/1720] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:11.355153 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 870/1721] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:11 2022] PUT /identity/v3/projects/9aacf82a037e4759a811b9c064510932/users/e0b39cbbaa4049af8d89e9a400b00abb/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:11.371997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613eb5d-363e-4b81-a0e1-d8e01d9f7b96 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:11.373329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613eb5d-363e-4b81-a0e1-d8e01d9f7b96 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:11.376491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613eb5d-363e-4b81-a0e1-d8e01d9f7b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:11.464772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f613eb5d-363e-4b81-a0e1-d8e01d9f7b96 None None] MFA Rules not processed for user `e0b39cbbaa4049af8d89e9a400b00abb`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:11.467634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f613eb5d-363e-4b81-a0e1-d8e01d9f7b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:11.505191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:11.617967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 852/1722] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:11.622199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f99ff-0ccb-4400-abe9-fdc21ef8cdff admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:11.774789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f99ff-0ccb-4400-abe9-fdc21ef8cdff admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:11.786661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699736f-d725-4b58-999a-012bd9ff3ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:11.788504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699736f-d725-4b58-999a-012bd9ff3ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:11.788504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699736f-d725-4b58-999a-012bd9ff3ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:11.789944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699736f-d725-4b58-999a-012bd9ff3ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:11.791512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699736f-d725-4b58-999a-012bd9ff3ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:11.797347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699736f-d725-4b58-999a-012bd9ff3ea1 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:11.950001 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 871/1723] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:11 2022] POST /identity/v3/projects => generated 369 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:11.955620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613eb5d-363e-4b81-a0e1-d8e01d9f7b96 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:11.982197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613eb5d-363e-4b81-a0e1-d8e01d9f7b96 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:12.195957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5c2d2-a2a8-47fe-9af1-7f4a59c68ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:12.199331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5c2d2-a2a8-47fe-9af1-7f4a59c68ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:12.202703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5c2d2-a2a8-47fe-9af1-7f4a59c68ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:12.208660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5c2d2-a2a8-47fe-9af1-7f4a59c68ad3 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:12.208660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5c2d2-a2a8-47fe-9af1-7f4a59c68ad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:12.211193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5c2d2-a2a8-47fe-9af1-7f4a59c68ad3 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:12.353926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 853/1724] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:11 2022] POST /identity/v3/users => generated 291 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:12.358539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699736f-d725-4b58-999a-012bd9ff3ea1 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:12.373110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699736f-d725-4b58-999a-012bd9ff3ea1 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:12.384695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f4628-0873-4daf-add8-6e5b71213821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:12.385419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f4628-0873-4daf-add8-6e5b71213821 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:12.385704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f4628-0873-4daf-add8-6e5b71213821 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:12.388467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f4628-0873-4daf-add8-6e5b71213821 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:12.392387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f4628-0873-4daf-add8-6e5b71213821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:12.398360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f4628-0873-4daf-add8-6e5b71213821 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:12.433249 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 872/1725] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:12.439370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5c2d2-a2a8-47fe-9af1-7f4a59c68ad3 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:12.455151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5c2d2-a2a8-47fe-9af1-7f4a59c68ad3 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:12.458504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:12.478205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63718fba-fa14-49da-a025-e956587ddd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:12.479264 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63718fba-fa14-49da-a025-e956587ddd19 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:12.479550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63718fba-fa14-49da-a025-e956587ddd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:12.479824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63718fba-fa14-49da-a025-e956587ddd19 admin admin] PATH_INFO: `/v3/projects/93a3d2f478cf47db80e7e15b92b0fc99/users/69934825cba94350a8dba1fee9271210/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:12.558827 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63718fba-fa14-49da-a025-e956587ddd19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a3d2f478cf47db80e7e15b92b0fc99, user_id=69934825cba94350a8dba1fee9271210, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:12.574026 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63718fba-fa14-49da-a025-e956587ddd19 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:12.679012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 854/1726] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:12 2022] PUT /identity/v3/projects/93a3d2f478cf47db80e7e15b92b0fc99/users/69934825cba94350a8dba1fee9271210/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:12.683043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f4628-0873-4daf-add8-6e5b71213821 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:12.709014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f4628-0873-4daf-add8-6e5b71213821 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:12.873719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd1b64-f223-4f53-afc6-436dde66a860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:12.873719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1b64-f223-4f53-afc6-436dde66a860 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:12.873719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1b64-f223-4f53-afc6-436dde66a860 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:12.873719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd1b64-f223-4f53-afc6-436dde66a860 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:12.880571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd1b64-f223-4f53-afc6-436dde66a860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:12.880571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd1b64-f223-4f53-afc6-436dde66a860 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:12.893904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 873/1727] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:12.898005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63718fba-fa14-49da-a025-e956587ddd19 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:12.909041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63718fba-fa14-49da-a025-e956587ddd19 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:12.926285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab390465-3f3d-4114-a9f8-6af52940ede2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:12.927198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab390465-3f3d-4114-a9f8-6af52940ede2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:12.927490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab390465-3f3d-4114-a9f8-6af52940ede2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:12.927773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab390465-3f3d-4114-a9f8-6af52940ede2 admin admin] PATH_INFO: `/v3/projects/93a3d2f478cf47db80e7e15b92b0fc99/users/69934825cba94350a8dba1fee9271210/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:12.935420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab390465-3f3d-4114-a9f8-6af52940ede2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a3d2f478cf47db80e7e15b92b0fc99, user_id=69934825cba94350a8dba1fee9271210, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:12.938655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab390465-3f3d-4114-a9f8-6af52940ede2 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:12.975373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 855/1728] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:12 2022] PUT /identity/v3/projects/93a3d2f478cf47db80e7e15b92b0fc99/users/69934825cba94350a8dba1fee9271210/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:12.988790 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc9051-4963-457a-be59-cd282ea82c51 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:12.989364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc9051-4963-457a-be59-cd282ea82c51 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:12.989442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc9051-4963-457a-be59-cd282ea82c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:13.100926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-badc9051-4963-457a-be59-cd282ea82c51 None None] MFA Rules not processed for user `69934825cba94350a8dba1fee9271210`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:13.102636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-badc9051-4963-457a-be59-cd282ea82c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:13.117067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:13.305082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 874/1729] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:12 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:13.314909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab390465-3f3d-4114-a9f8-6af52940ede2 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:13.316722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ab390465-3f3d-4114-a9f8-6af52940ede2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:13.499065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab390465-3f3d-4114-a9f8-6af52940ede2 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:13.504690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88752443-ecd0-46d6-8c77-4b63b2772697 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:13.505573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88752443-ecd0-46d6-8c77-4b63b2772697 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:13.506005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88752443-ecd0-46d6-8c77-4b63b2772697 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:13.506429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88752443-ecd0-46d6-8c77-4b63b2772697 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:13.518135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88752443-ecd0-46d6-8c77-4b63b2772697 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:13.519885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88752443-ecd0-46d6-8c77-4b63b2772697 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:13.532245 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 856/1730] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:13 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:13.703077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:14.534828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddcedb-49e4-4fac-a945-9fcd4902496b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:14.535305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddcedb-49e4-4fac-a945-9fcd4902496b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:14.535404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddcedb-49e4-4fac-a945-9fcd4902496b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:14.579097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f3ddcedb-49e4-4fac-a945-9fcd4902496b None None] MFA Rules not processed for user `2de523abbc7b4f8b972dc97b8a013237`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:14.580509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f3ddcedb-49e4-4fac-a945-9fcd4902496b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:14.662154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 875/1731] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:15.745861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88752443-ecd0-46d6-8c77-4b63b2772697 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:15.747518 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-88752443-ecd0-46d6-8c77-4b63b2772697 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:15.858338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddcedb-49e4-4fac-a945-9fcd4902496b None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:15.859977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f3ddcedb-49e4-4fac-a945-9fcd4902496b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:15.911791 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88752443-ecd0-46d6-8c77-4b63b2772697 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:15.918798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997583b-529f-4d6a-85b8-75ad560e6e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:15.920166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997583b-529f-4d6a-85b8-75ad560e6e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:15.920639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997583b-529f-4d6a-85b8-75ad560e6e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:15.921444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997583b-529f-4d6a-85b8-75ad560e6e42 admin admin] PATH_INFO: `/v3/users/6367ef477233433a9fab0dab1dd734f7` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:15.927218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997583b-529f-4d6a-85b8-75ad560e6e42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6367ef477233433a9fab0dab1dd734f7)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:15.934085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997583b-529f-4d6a-85b8-75ad560e6e42 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:16.097381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddcedb-49e4-4fac-a945-9fcd4902496b None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:16.110343 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c02be-9742-4101-8c3c-ef86f0954bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:16.113206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c02be-9742-4101-8c3c-ef86f0954bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:16.113681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c02be-9742-4101-8c3c-ef86f0954bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:16.114103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c02be-9742-4101-8c3c-ef86f0954bd5 admin admin] PATH_INFO: `/v3/users/f81961656d094e8aa763228a335f5733` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:16.117924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c02be-9742-4101-8c3c-ef86f0954bd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f81961656d094e8aa763228a335f5733)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:16.119287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c02be-9742-4101-8c3c-ef86f0954bd5 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:16.312438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8997583b-529f-4d6a-85b8-75ad560e6e42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6367ef477233433a9fab0dab1dd734f7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:16.331850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8997583b-529f-4d6a-85b8-75ad560e6e42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6367ef477233433a9fab0dab1dd734f7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:16.332691 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8997583b-529f-4d6a-85b8-75ad560e6e42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6367ef477233433a9fab0dab1dd734f7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:16.371604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:16.387024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8997583b-529f-4d6a-85b8-75ad560e6e42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6367ef477233433a9fab0dab1dd734f7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:16.389231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8997583b-529f-4d6a-85b8-75ad560e6e42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6367ef477233433a9fab0dab1dd734f7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:16.410353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f33c02be-9742-4101-8c3c-ef86f0954bd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f81961656d094e8aa763228a335f5733'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:16.411070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f33c02be-9742-4101-8c3c-ef86f0954bd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f81961656d094e8aa763228a335f5733'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:16.414168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:16.439859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f33c02be-9742-4101-8c3c-ef86f0954bd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f81961656d094e8aa763228a335f5733'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:16.453809 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f33c02be-9742-4101-8c3c-ef86f0954bd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f81961656d094e8aa763228a335f5733'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:16.498184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f33c02be-9742-4101-8c3c-ef86f0954bd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f81961656d094e8aa763228a335f5733'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:16.528613 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 857/1732] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:15 2022] DELETE /identity/v3/users/6367ef477233433a9fab0dab1dd734f7 => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:16.530175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569eaaa-8dfc-4d59-a737-e7860c6dad88 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:16.531742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569eaaa-8dfc-4d59-a737-e7860c6dad88 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:16.532844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569eaaa-8dfc-4d59-a737-e7860c6dad88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:16.579409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b569eaaa-8dfc-4d59-a737-e7860c6dad88 None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:16.580676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b569eaaa-8dfc-4d59-a737-e7860c6dad88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:16.618347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 876/1733] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:15 2022] DELETE /identity/v3/users/f81961656d094e8aa763228a335f5733 => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:16.620037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff4096-43d0-4d10-9389-3dcbb5565438 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:16.620472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff4096-43d0-4d10-9389-3dcbb5565438 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:16.621549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff4096-43d0-4d10-9389-3dcbb5565438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:16.675110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-8aff4096-43d0-4d10-9389-3dcbb5565438 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:16.676373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8aff4096-43d0-4d10-9389-3dcbb5565438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:16.729254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 858/1734] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 18:56:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:16.731206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f947d-5cf8-4776-b2b9-b6710ebba9ac None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:16.742409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f947d-5cf8-4776-b2b9-b6710ebba9ac None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:16.742409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f947d-5cf8-4776-b2b9-b6710ebba9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:16.753258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-179f947d-5cf8-4776-b2b9-b6710ebba9ac None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:16.754510 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-179f947d-5cf8-4776-b2b9-b6710ebba9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:16.881972 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 859/1735] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:16.888652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 877/1736] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:16.923940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f947d-5cf8-4776-b2b9-b6710ebba9ac None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:16.926062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-179f947d-5cf8-4776-b2b9-b6710ebba9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:16.953828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff4096-43d0-4d10-9389-3dcbb5565438 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:16.956370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8aff4096-43d0-4d10-9389-3dcbb5565438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:17.073440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f947d-5cf8-4776-b2b9-b6710ebba9ac None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:17.076086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff4096-43d0-4d10-9389-3dcbb5565438 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:17.087178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c99aa-5cff-4900-abfd-4ea97c734b57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:17.089077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c99aa-5cff-4900-abfd-4ea97c734b57 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:17.089077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c99aa-5cff-4900-abfd-4ea97c734b57 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:17.089077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c99aa-5cff-4900-abfd-4ea97c734b57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:17.106139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e5c18-efc0-4aa6-b349-7758a5c9b412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:17.106139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e5c18-efc0-4aa6-b349-7758a5c9b412 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:17.106139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e5c18-efc0-4aa6-b349-7758a5c9b412 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:17.106139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e5c18-efc0-4aa6-b349-7758a5c9b412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:17.111545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c99aa-5cff-4900-abfd-4ea97c734b57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:17.113746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c99aa-5cff-4900-abfd-4ea97c734b57 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:17.126439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e5c18-efc0-4aa6-b349-7758a5c9b412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:17.129083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e5c18-efc0-4aa6-b349-7758a5c9b412 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:17.141455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 860/1737] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:17.154850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 878/1738] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:17.241019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c99aa-5cff-4900-abfd-4ea97c734b57 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:17.242581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f99c99aa-5cff-4900-abfd-4ea97c734b57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:17.383943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c99aa-5cff-4900-abfd-4ea97c734b57 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:17.398205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18453d89-4260-451c-b36a-d1294af7cd7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:17.399043 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18453d89-4260-451c-b36a-d1294af7cd7d service swift] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:17.401013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18453d89-4260-451c-b36a-d1294af7cd7d service swift] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:17.401463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18453d89-4260-451c-b36a-d1294af7cd7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:17.563670 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18453d89-4260-451c-b36a-d1294af7cd7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:17.565823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18453d89-4260-451c-b36a-d1294af7cd7d service swift] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:17.599068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 861/1739] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 18:56:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:56:18.383545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:19.149168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0eabca-22c9-4a1d-9855-57ae55e90fe6 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:19.149168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0eabca-22c9-4a1d-9855-57ae55e90fe6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:19.149168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0eabca-22c9-4a1d-9855-57ae55e90fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:19.149168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ed0eabca-22c9-4a1d-9855-57ae55e90fe6 None None] MFA Rules not processed for user `f4e05d75cd1142e18d7518b78edd88fa`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:19.149168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ed0eabca-22c9-4a1d-9855-57ae55e90fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:19.149168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 879/1740] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Jun 16 18:56:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:19.153135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18453d89-4260-451c-b36a-d1294af7cd7d service swift] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:19.153135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18453d89-4260-451c-b36a-d1294af7cd7d service swift] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:19.153135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd17f88-7e82-4271-a92a-99554b1c7eb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:19.153135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17f88-7e82-4271-a92a-99554b1c7eb6 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:19.154601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17f88-7e82-4271-a92a-99554b1c7eb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:19.154601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd17f88-7e82-4271-a92a-99554b1c7eb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:19.167165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd17f88-7e82-4271-a92a-99554b1c7eb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:19.168977 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd17f88-7e82-4271-a92a-99554b1c7eb6 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:19.181999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 862/1741] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:19.930410 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0eabca-22c9-4a1d-9855-57ae55e90fe6 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:19.938183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ed0eabca-22c9-4a1d-9855-57ae55e90fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:19.975550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0eabca-22c9-4a1d-9855-57ae55e90fe6 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:20.000953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d32372-afa6-4e43-9b73-e35dbd3ac2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:20.005400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d32372-afa6-4e43-9b73-e35dbd3ac2ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:20.005400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d32372-afa6-4e43-9b73-e35dbd3ac2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:20.005400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d32372-afa6-4e43-9b73-e35dbd3ac2ca admin admin] PATH_INFO: `/v3/users/e0b39cbbaa4049af8d89e9a400b00abb` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:20.011298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d32372-afa6-4e43-9b73-e35dbd3ac2ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b39cbbaa4049af8d89e9a400b00abb)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:20.020847 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d32372-afa6-4e43-9b73-e35dbd3ac2ca admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:20.053765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd17f88-7e82-4271-a92a-99554b1c7eb6 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:20.055349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-dfd17f88-7e82-4271-a92a-99554b1c7eb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:20.086564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd17f88-7e82-4271-a92a-99554b1c7eb6 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:20.111912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:20.114545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:20.114545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:20.114545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] PATH_INFO: `/v3/projects/6c22b67eb39f4392baa0b135b7f58dae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:20.116317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c22b67eb39f4392baa0b135b7f58dae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:20.121871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:20.353715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b4d32372-afa6-4e43-9b73-e35dbd3ac2ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b39cbbaa4049af8d89e9a400b00abb'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:20.357473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b4d32372-afa6-4e43-9b73-e35dbd3ac2ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b39cbbaa4049af8d89e9a400b00abb'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:20.386216 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c22b67eb39f4392baa0b135b7f58dae', 'request_id': 'req-0eda7c74-75cb-404a-9da4-8757c8ce27d5'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:20.427952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b4d32372-afa6-4e43-9b73-e35dbd3ac2ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b39cbbaa4049af8d89e9a400b00abb'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:20.431338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b4d32372-afa6-4e43-9b73-e35dbd3ac2ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b39cbbaa4049af8d89e9a400b00abb'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:20.438661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:20.497231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c22b67eb39f4392baa0b135b7f58dae', 'request_id': 'req-0eda7c74-75cb-404a-9da4-8757c8ce27d5'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:20.501686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b4d32372-afa6-4e43-9b73-e35dbd3ac2ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b39cbbaa4049af8d89e9a400b00abb'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:20.510181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c22b67eb39f4392baa0b135b7f58dae', 'request_id': 'req-0eda7c74-75cb-404a-9da4-8757c8ce27d5'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:20.551637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] The token cache is being invalidate because project 6c22b67eb39f4392baa0b135b7f58dae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:20.552107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:20.553335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:20.555388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 863/1742] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:20 2022] DELETE /identity/v3/projects/6c22b67eb39f4392baa0b135b7f58dae => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:20.564407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:20.566133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:20.614208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 880/1743] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:19 2022] DELETE /identity/v3/users/e0b39cbbaa4049af8d89e9a400b00abb => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:20.618442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266587-acf2-4636-ad9b-bb26f5792d06 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:20.618931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266587-acf2-4636-ad9b-bb26f5792d06 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:20.619360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266587-acf2-4636-ad9b-bb26f5792d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:20.693488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0b266587-acf2-4636-ad9b-bb26f5792d06 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:20.695077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0b266587-acf2-4636-ad9b-bb26f5792d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:20.800476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:20.909547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda7c74-75cb-404a-9da4-8757c8ce27d5 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:21.009019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 881/1744] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:21.015855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b266587-acf2-4636-ad9b-bb26f5792d06 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:21.021356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0b266587-acf2-4636-ad9b-bb26f5792d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:21.049290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b266587-acf2-4636-ad9b-bb26f5792d06 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:21.061922 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:21.062813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:21.063099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:21.063375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] PATH_INFO: `/v3/projects/93ca6ea99cdb4f74b3372635fb2858ba` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:21.066148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93ca6ea99cdb4f74b3372635fb2858ba)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:21.076382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:21.089973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:21.090874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:21.091163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:21.091446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] PATH_INFO: `/v3/users/0f7cca3c7f2e48608d5c31ccde963c39` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:21.094214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f7cca3c7f2e48608d5c31ccde963c39)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:21.096197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:21.235449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93ca6ea99cdb4f74b3372635fb2858ba', 'request_id': 'req-5c1d8524-0967-463b-addb-ef170b813a84'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:21.301435 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f7cca3c7f2e48608d5c31ccde963c39'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:21.303199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93ca6ea99cdb4f74b3372635fb2858ba', 'request_id': 'req-5c1d8524-0967-463b-addb-ef170b813a84'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:21.310068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f7cca3c7f2e48608d5c31ccde963c39'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:21.311806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f7cca3c7f2e48608d5c31ccde963c39'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:21.312726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93ca6ea99cdb4f74b3372635fb2858ba', 'request_id': 'req-5c1d8524-0967-463b-addb-ef170b813a84'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:21.337302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] The token cache is being invalidate because project 93ca6ea99cdb4f74b3372635fb2858ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:21.339436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:21.350203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:21.353668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f7cca3c7f2e48608d5c31ccde963c39'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:21.354896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 882/1745] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:21 2022] DELETE /identity/v3/projects/93ca6ea99cdb4f74b3372635fb2858ba => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:21.356397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:21.358099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f7cca3c7f2e48608d5c31ccde963c39'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:21.372601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:21.471988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 864/1746] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:20 2022] DELETE /identity/v3/users/0f7cca3c7f2e48608d5c31ccde963c39 => generated 0 bytes in 908 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:21.473219 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:21.475799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:21.482178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d8524-0967-463b-addb-ef170b813a84 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:21.663879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58720cc-5389-40c9-bf62-6e2d7a064433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:21.666332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58720cc-5389-40c9-bf62-6e2d7a064433 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:21.667199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58720cc-5389-40c9-bf62-6e2d7a064433 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:21.669550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58720cc-5389-40c9-bf62-6e2d7a064433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:21.676030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c58720cc-5389-40c9-bf62-6e2d7a064433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:21.696183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c1e2d-399d-483f-9d19-963a692994a0 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:21.719899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe1a039-10a1-4523-ae50-aa31517c2d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:21.723402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1a039-10a1-4523-ae50-aa31517c2d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:21.725910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58720cc-5389-40c9-bf62-6e2d7a064433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:21.729465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1a039-10a1-4523-ae50-aa31517c2d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:21.732022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1a039-10a1-4523-ae50-aa31517c2d9a admin admin] PATH_INFO: `/v3/users/ae97803f6ad14ddf8b689ca319bb28ff` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:21.734041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58720cc-5389-40c9-bf62-6e2d7a064433 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:21.737968 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1a039-10a1-4523-ae50-aa31517c2d9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae97803f6ad14ddf8b689ca319bb28ff)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:21.742862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe1a039-10a1-4523-ae50-aa31517c2d9a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:21.766525 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 883/1747] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:21.949200 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:22.034211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5fe1a039-10a1-4523-ae50-aa31517c2d9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae97803f6ad14ddf8b689ca319bb28ff'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:22.050829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5fe1a039-10a1-4523-ae50-aa31517c2d9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae97803f6ad14ddf8b689ca319bb28ff'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:22.052527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5fe1a039-10a1-4523-ae50-aa31517c2d9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae97803f6ad14ddf8b689ca319bb28ff'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:22.087514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5fe1a039-10a1-4523-ae50-aa31517c2d9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae97803f6ad14ddf8b689ca319bb28ff'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:22.088331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5fe1a039-10a1-4523-ae50-aa31517c2d9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae97803f6ad14ddf8b689ca319bb28ff'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:22.235772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 865/1748] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:21 2022] DELETE /identity/v3/users/ae97803f6ad14ddf8b689ca319bb28ff => generated 0 bytes in 758 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:22.250332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc11253-9697-4458-aff4-11c039ea0108 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:22.250634 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc11253-9697-4458-aff4-11c039ea0108 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:22.250989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc11253-9697-4458-aff4-11c039ea0108 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:22.252451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 884/1749] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 18:56:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:56:22.317849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07394519-7e48-4208-b81d-e3fd351d36e4 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:22.318144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07394519-7e48-4208-b81d-e3fd351d36e4 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:22.318597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07394519-7e48-4208-b81d-e3fd351d36e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:22.366968 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-07394519-7e48-4208-b81d-e3fd351d36e4 None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:22.369509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-07394519-7e48-4208-b81d-e3fd351d36e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:22.469232 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:22.574006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 866/1750] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 18:56:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:23.466659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc11253-9697-4458-aff4-11c039ea0108 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:23.480184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7fc11253-9697-4458-aff4-11c039ea0108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:23.528871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc11253-9697-4458-aff4-11c039ea0108 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:23.560595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5541f9-255f-4465-b080-d6ab6cd13279 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:23.567302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5541f9-255f-4465-b080-d6ab6cd13279 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:23.569138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5541f9-255f-4465-b080-d6ab6cd13279 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:23.571844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5541f9-255f-4465-b080-d6ab6cd13279 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:23.601919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5541f9-255f-4465-b080-d6ab6cd13279 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:23.605698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5541f9-255f-4465-b080-d6ab6cd13279 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:23.666571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 885/1751] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:23.917977 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07394519-7e48-4208-b81d-e3fd351d36e4 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:23.924651 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-07394519-7e48-4208-b81d-e3fd351d36e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:24.073759 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07394519-7e48-4208-b81d-e3fd351d36e4 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:24.086690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:24.086690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:24.086690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:24.086690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] PATH_INFO: `/v3/projects/dc00b27d9949434189d02d2c078fa22f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:24.087143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc00b27d9949434189d02d2c078fa22f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:24.088508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:24.154007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:24.280909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc00b27d9949434189d02d2c078fa22f', 'request_id': 'req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:24.378841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc00b27d9949434189d02d2c078fa22f', 'request_id': 'req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:24.386172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc00b27d9949434189d02d2c078fa22f', 'request_id': 'req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:24.415158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] The token cache is being invalidate because project dc00b27d9949434189d02d2c078fa22f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:24.415462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:24.416316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-14f7d0b8-f75e-4eeb-a50f-57364e79dfe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:24.418256 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 867/1752] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:23 2022] DELETE /identity/v3/projects/dc00b27d9949434189d02d2c078fa22f => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:24.425037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3dc75-72aa-41f7-9d90-817aae63a95c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:24.425455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3dc75-72aa-41f7-9d90-817aae63a95c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:24.425944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3dc75-72aa-41f7-9d90-817aae63a95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:24.455102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-31f3dc75-72aa-41f7-9d90-817aae63a95c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:24.456658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-31f3dc75-72aa-41f7-9d90-817aae63a95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:24.538916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 886/1753] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:24.549782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a9638-0ef9-42f6-a929-a75b232a7f58 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:24.550633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a9638-0ef9-42f6-a929-a75b232a7f58 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:24.552049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a9638-0ef9-42f6-a929-a75b232a7f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:24.591387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-cc7a9638-0ef9-42f6-a929-a75b232a7f58 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:24.593669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-cc7a9638-0ef9-42f6-a929-a75b232a7f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:24.618590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 868/1754] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:24.618771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3dc75-72aa-41f7-9d90-817aae63a95c None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:24.629689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3dc75-72aa-41f7-9d90-817aae63a95c None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:24.635961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f62539-118d-49a8-a0b9-acbac477b8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:24.636947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f62539-118d-49a8-a0b9-acbac477b8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:24.637487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f62539-118d-49a8-a0b9-acbac477b8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:24.638438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f62539-118d-49a8-a0b9-acbac477b8e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:24.639428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f62539-118d-49a8-a0b9-acbac477b8e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:56:24.640108 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f62539-118d-49a8-a0b9-acbac477b8e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:24.642421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f62539-118d-49a8-a0b9-acbac477b8e6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:24.663625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 887/1755] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:56:24.667872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a9638-0ef9-42f6-a929-a75b232a7f58 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:24.679849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a9638-0ef9-42f6-a929-a75b232a7f58 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:24.695743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47585f49-22e4-40e8-9d5e-408b43a9eda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:24.697828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47585f49-22e4-40e8-9d5e-408b43a9eda6 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:24.698417 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47585f49-22e4-40e8-9d5e-408b43a9eda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:24.699122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47585f49-22e4-40e8-9d5e-408b43a9eda6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:24.700300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47585f49-22e4-40e8-9d5e-408b43a9eda6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:24.703641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47585f49-22e4-40e8-9d5e-408b43a9eda6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:24.714834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f62539-118d-49a8-a0b9-acbac477b8e6 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:24.717446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f7f62539-118d-49a8-a0b9-acbac477b8e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:24.739926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f62539-118d-49a8-a0b9-acbac477b8e6 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:24.771003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:24.772256 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:24.772680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:24.773060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] PATH_INFO: `/v3/projects/9aacf82a037e4759a811b9c064510932` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:24.775032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aacf82a037e4759a811b9c064510932)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:24.788793 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:24.807339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 869/1756] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:24 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:24.808572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47585f49-22e4-40e8-9d5e-408b43a9eda6 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:24.813786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-47585f49-22e4-40e8-9d5e-408b43a9eda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:24.973349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aacf82a037e4759a811b9c064510932', 'request_id': 'req-0c17742c-1126-48e6-be01-d4c9c91587e8'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.057312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aacf82a037e4759a811b9c064510932', 'request_id': 'req-0c17742c-1126-48e6-be01-d4c9c91587e8'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.060011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aacf82a037e4759a811b9c064510932', 'request_id': 'req-0c17742c-1126-48e6-be01-d4c9c91587e8'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.104105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] The token cache is being invalidate because project 9aacf82a037e4759a811b9c064510932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:25.106121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.107513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.113604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 888/1757] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:24 2022] DELETE /identity/v3/projects/9aacf82a037e4759a811b9c064510932 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:25.114605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:25.120954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:25.174233 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:25.261351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17742c-1126-48e6-be01-d4c9c91587e8 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:25.274221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47585f49-22e4-40e8-9d5e-408b43a9eda6 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:25.277540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3338198-11bd-4afb-9c2a-076982cebc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:25.282528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3338198-11bd-4afb-9c2a-076982cebc97 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:25.282528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3338198-11bd-4afb-9c2a-076982cebc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:25.282528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3338198-11bd-4afb-9c2a-076982cebc97 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:25.282528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3338198-11bd-4afb-9c2a-076982cebc97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:25.284677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3338198-11bd-4afb-9c2a-076982cebc97 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:25.290113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:25.291022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:25.291283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:25.291560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] PATH_INFO: `/v3/projects/88fc6ce4c4e0474987ebedcfc10ce52a` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:25.292900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=88fc6ce4c4e0474987ebedcfc10ce52a)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:25.304162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:25.427695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 889/1758] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:25 2022] POST /identity/v3/users => generated 283 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:25.429175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3338198-11bd-4afb-9c2a-076982cebc97 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:25.432030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b3338198-11bd-4afb-9c2a-076982cebc97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:25.456648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3338198-11bd-4afb-9c2a-076982cebc97 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:25.469801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:25.473399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:25.473699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:25.481150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] PATH_INFO: `/v3/users/69934825cba94350a8dba1fee9271210` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:25.481150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69934825cba94350a8dba1fee9271210)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:25.484156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:25.508381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88fc6ce4c4e0474987ebedcfc10ce52a', 'request_id': 'req-f1e2345d-538f-4936-b790-85885dbf24aa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.594247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88fc6ce4c4e0474987ebedcfc10ce52a', 'request_id': 'req-f1e2345d-538f-4936-b790-85885dbf24aa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.607263 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88fc6ce4c4e0474987ebedcfc10ce52a', 'request_id': 'req-f1e2345d-538f-4936-b790-85885dbf24aa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.638794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] The token cache is being invalidate because project 88fc6ce4c4e0474987ebedcfc10ce52a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:25.639086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.640455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.653036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 870/1759] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:24 2022] DELETE /identity/v3/projects/88fc6ce4c4e0474987ebedcfc10ce52a => generated 0 bytes in 845 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:25.654016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:25.658612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:25.696397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69934825cba94350a8dba1fee9271210'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.697801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69934825cba94350a8dba1fee9271210'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.698254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:25.720121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69934825cba94350a8dba1fee9271210'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.721217 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69934825cba94350a8dba1fee9271210'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.744026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69934825cba94350a8dba1fee9271210'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:25.818578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2345d-538f-4936-b790-85885dbf24aa admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:25.831784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1e499-7776-4fe8-a9c3-6efdf008459b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:25.832745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1e499-7776-4fe8-a9c3-6efdf008459b admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:25.833202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1e499-7776-4fe8-a9c3-6efdf008459b admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:25.838598 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1e499-7776-4fe8-a9c3-6efdf008459b admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:25.840182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1e499-7776-4fe8-a9c3-6efdf008459b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:25.846392 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1e499-7776-4fe8-a9c3-6efdf008459b admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:25.859469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 871/1760] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:25.861048 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e4b45-b237-49a1-b73b-bd64d3b1c712 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:25.861401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e4b45-b237-49a1-b73b-bd64d3b1c712 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:25.861671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e4b45-b237-49a1-b73b-bd64d3b1c712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:25.875294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 890/1761] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:25 2022] DELETE /identity/v3/users/69934825cba94350a8dba1fee9271210 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:25.880800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:25.880800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:25.884319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1c7e4b45-b237-49a1-b73b-bd64d3b1c712 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:25.886202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1c7e4b45-b237-49a1-b73b-bd64d3b1c712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:25.966928 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 872/1762] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:25.974594 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0820a-f4e3-437f-97fc-1fb4c27c7e1c None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:25.979671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0820a-f4e3-437f-97fc-1fb4c27c7e1c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:25.979671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0820a-f4e3-437f-97fc-1fb4c27c7e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:26.040528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-69e0820a-f4e3-437f-97fc-1fb4c27c7e1c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:26.044196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-69e0820a-f4e3-437f-97fc-1fb4c27c7e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:26.071499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 873/1763] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:26.072925 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0820a-f4e3-437f-97fc-1fb4c27c7e1c None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:26.073975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ad80b-5f25-4c74-92dc-50c2de7b88d6 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:26.091097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a216ecf-69a4-4459-aa2d-7bb7e3d1d980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:26.091911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a216ecf-69a4-4459-aa2d-7bb7e3d1d980 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:26.092305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a216ecf-69a4-4459-aa2d-7bb7e3d1d980 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:26.092733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a216ecf-69a4-4459-aa2d-7bb7e3d1d980 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:26.093613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a216ecf-69a4-4459-aa2d-7bb7e3d1d980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:26.095587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a216ecf-69a4-4459-aa2d-7bb7e3d1d980 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:26.115286 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0820a-f4e3-437f-97fc-1fb4c27c7e1c None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:26.122303 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 891/1764] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:26.123398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a216ecf-69a4-4459-aa2d-7bb7e3d1d980 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:26.133176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea555d02-d467-44f3-9cd7-72816d5f6020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:26.134187 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea555d02-d467-44f3-9cd7-72816d5f6020 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:26.135037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea555d02-d467-44f3-9cd7-72816d5f6020 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:26.135912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea555d02-d467-44f3-9cd7-72816d5f6020 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:26.137821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea555d02-d467-44f3-9cd7-72816d5f6020 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:56:26.138148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea555d02-d467-44f3-9cd7-72816d5f6020 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:26.141896 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea555d02-d467-44f3-9cd7-72816d5f6020 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:26.149117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a216ecf-69a4-4459-aa2d-7bb7e3d1d980 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:26.165228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc583a8a-ba8f-4a4d-96f7-0c4e50f71633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:26.166511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc583a8a-ba8f-4a4d-96f7-0c4e50f71633 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:26.170170 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc583a8a-ba8f-4a4d-96f7-0c4e50f71633 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:26.170170 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc583a8a-ba8f-4a4d-96f7-0c4e50f71633 admin admin] PATH_INFO: `/v3/projects/37258dbf12af4505bbe44a542a261a61/users/3bbf01dcd9b748eb9e8f452b8aa28d2c/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:26.213355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 874/1765] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:56:26.215329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea555d02-d467-44f3-9cd7-72816d5f6020 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:26.215454 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc583a8a-ba8f-4a4d-96f7-0c4e50f71633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37258dbf12af4505bbe44a542a261a61, user_id=3bbf01dcd9b748eb9e8f452b8aa28d2c, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:26.218601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc583a8a-ba8f-4a4d-96f7-0c4e50f71633 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:26.239310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea555d02-d467-44f3-9cd7-72816d5f6020 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:26.257572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86333f4-0bae-4ab1-b6d2-ed4d4631732e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:26.259568 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86333f4-0bae-4ab1-b6d2-ed4d4631732e admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:26.259864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86333f4-0bae-4ab1-b6d2-ed4d4631732e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:26.260770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86333f4-0bae-4ab1-b6d2-ed4d4631732e admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:26.261632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86333f4-0bae-4ab1-b6d2-ed4d4631732e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:26.266153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86333f4-0bae-4ab1-b6d2-ed4d4631732e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:26.286759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 892/1766] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:26 2022] PUT /identity/v3/projects/37258dbf12af4505bbe44a542a261a61/users/3bbf01dcd9b748eb9e8f452b8aa28d2c/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:26.300349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5f8cc-1fa8-417f-9f55-0124d40709ac None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:26.301649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5f8cc-1fa8-417f-9f55-0124d40709ac None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:26.302579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5f8cc-1fa8-417f-9f55-0124d40709ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:26.342372 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-53f5f8cc-1fa8-417f-9f55-0124d40709ac None None] MFA Rules not processed for user `3bbf01dcd9b748eb9e8f452b8aa28d2c`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:26.344698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-53f5f8cc-1fa8-417f-9f55-0124d40709ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:26.375156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 875/1767] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:26 2022] POST /identity/v3/projects => generated 357 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:26.376880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86333f4-0bae-4ab1-b6d2-ed4d4631732e admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:26.446022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:26.540879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 893/1768] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:26.542037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5f8cc-1fa8-417f-9f55-0124d40709ac None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:26.568458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86333f4-0bae-4ab1-b6d2-ed4d4631732e admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:26.604601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74330b40-0e75-429b-a5eb-3a12dab008bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:26.605323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74330b40-0e75-429b-a5eb-3a12dab008bf admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:26.605612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74330b40-0e75-429b-a5eb-3a12dab008bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:26.605886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74330b40-0e75-429b-a5eb-3a12dab008bf admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:26.606845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74330b40-0e75-429b-a5eb-3a12dab008bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:26.616585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74330b40-0e75-429b-a5eb-3a12dab008bf admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:26.669422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5f8cc-1fa8-417f-9f55-0124d40709ac None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:26.690670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911a2b1-6117-4581-92e9-f44b56b02a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:26.691357 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911a2b1-6117-4581-92e9-f44b56b02a29 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:26.691644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911a2b1-6117-4581-92e9-f44b56b02a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:26.691921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911a2b1-6117-4581-92e9-f44b56b02a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:26.692621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911a2b1-6117-4581-92e9-f44b56b02a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:26.694567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911a2b1-6117-4581-92e9-f44b56b02a29 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:26.769300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 876/1769] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:26 2022] POST /identity/v3/users => generated 279 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:26.772678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74330b40-0e75-429b-a5eb-3a12dab008bf admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:26.811381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74330b40-0e75-429b-a5eb-3a12dab008bf admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:26.821962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 894/1770] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:26 2022] POST /identity/v3/projects => generated 363 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:26.823055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911a2b1-6117-4581-92e9-f44b56b02a29 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:26.824893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633efbfc-fd42-4362-9a9d-d3ad8806d240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:26.825926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633efbfc-fd42-4362-9a9d-d3ad8806d240 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:26.826326 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633efbfc-fd42-4362-9a9d-d3ad8806d240 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:26.826718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633efbfc-fd42-4362-9a9d-d3ad8806d240 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:26.828255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633efbfc-fd42-4362-9a9d-d3ad8806d240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:26.834601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633efbfc-fd42-4362-9a9d-d3ad8806d240 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:26.866246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 877/1771] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:26.870862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633efbfc-fd42-4362-9a9d-d3ad8806d240 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:26.894841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633efbfc-fd42-4362-9a9d-d3ad8806d240 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:26.902661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:26.981870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911a2b1-6117-4581-92e9-f44b56b02a29 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:27.020409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279cff5-7d24-40ee-a486-982e640f7345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:27.021132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279cff5-7d24-40ee-a486-982e640f7345 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:27.021456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279cff5-7d24-40ee-a486-982e640f7345 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:27.021741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279cff5-7d24-40ee-a486-982e640f7345 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:27.022467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279cff5-7d24-40ee-a486-982e640f7345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:27.025556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1279cff5-7d24-40ee-a486-982e640f7345 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:27.047103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074352b-c61d-41e7-b927-11a6a264cf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:27.048005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074352b-c61d-41e7-b927-11a6a264cf04 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:27.048410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074352b-c61d-41e7-b927-11a6a264cf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:27.048815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074352b-c61d-41e7-b927-11a6a264cf04 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:27.050460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074352b-c61d-41e7-b927-11a6a264cf04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:27.052389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074352b-c61d-41e7-b927-11a6a264cf04 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:27.081600 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 878/1772] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:27.083836 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca4a30-3a1b-44c3-88b3-a97e0f20d30f None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:27.085014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca4a30-3a1b-44c3-88b3-a97e0f20d30f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:27.085565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca4a30-3a1b-44c3-88b3-a97e0f20d30f None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:27.087368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 879/1773] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 18:56:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:56:27.088470 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca4a30-3a1b-44c3-88b3-a97e0f20d30f None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:27.107920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca4a30-3a1b-44c3-88b3-a97e0f20d30f None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:27.113977 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dfeb7-a7b3-4a6a-8cb1-253bbb830319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:27.115462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dfeb7-a7b3-4a6a-8cb1-253bbb830319 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:27.115989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dfeb7-a7b3-4a6a-8cb1-253bbb830319 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:27.116533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dfeb7-a7b3-4a6a-8cb1-253bbb830319 admin admin] PATH_INFO: `/v3/projects/30385f3613104912a0f18f7e153950fe/users/27fb9df11b87426795f9f12d5c2eb86a/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:27.145140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531dfeb7-a7b3-4a6a-8cb1-253bbb830319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30385f3613104912a0f18f7e153950fe, user_id=27fb9df11b87426795f9f12d5c2eb86a, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:27.148392 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531dfeb7-a7b3-4a6a-8cb1-253bbb830319 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:27.180058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 895/1774] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:26 2022] POST /identity/v3/users => generated 288 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:27.181662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1ed16-665f-437a-bb43-0f044101325f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:27.181995 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1ed16-665f-437a-bb43-0f044101325f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:27.182261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac1ed16-665f-437a-bb43-0f044101325f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:27.215184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3ac1ed16-665f-437a-bb43-0f044101325f None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:27.216420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3ac1ed16-665f-437a-bb43-0f044101325f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:27.246652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 880/1775] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:27 2022] PUT /identity/v3/projects/30385f3613104912a0f18f7e153950fe/users/27fb9df11b87426795f9f12d5c2eb86a/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:27.247465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dfeb7-a7b3-4a6a-8cb1-253bbb830319 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:27.399502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 896/1776] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 18:56:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:27.401015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e46ba-6239-4b65-86ad-b44efac9eeb5 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:27.402899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e46ba-6239-4b65-86ad-b44efac9eeb5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:27.403997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e46ba-6239-4b65-86ad-b44efac9eeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:27.406398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dfeb7-a7b3-4a6a-8cb1-253bbb830319 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:27.421961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5289f8e-46fc-48a1-ab02-cbf1c0b562df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:27.426631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289f8e-46fc-48a1-ab02-cbf1c0b562df admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:27.426631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289f8e-46fc-48a1-ab02-cbf1c0b562df admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:27.426631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5289f8e-46fc-48a1-ab02-cbf1c0b562df admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:27.427139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5289f8e-46fc-48a1-ab02-cbf1c0b562df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:27.431339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5289f8e-46fc-48a1-ab02-cbf1c0b562df admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:27.485162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-eb5e46ba-6239-4b65-86ad-b44efac9eeb5 None None] MFA Rules not processed for user `27fb9df11b87426795f9f12d5c2eb86a`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:27.486990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-eb5e46ba-6239-4b65-86ad-b44efac9eeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:27.487635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 881/1777] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:27.488731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5289f8e-46fc-48a1-ab02-cbf1c0b562df admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:27.498341 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b5289f8e-46fc-48a1-ab02-cbf1c0b562df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:27.529727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5289f8e-46fc-48a1-ab02-cbf1c0b562df admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:27.553296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:27.554166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:27.554452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:27.554729 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] PATH_INFO: `/v3/projects/93a3d2f478cf47db80e7e15b92b0fc99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:27.556407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a3d2f478cf47db80e7e15b92b0fc99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:27.558411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:27.571260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:27.614531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 897/1778] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:27 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:27.615507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e46ba-6239-4b65-86ad-b44efac9eeb5 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:27.644948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e46ba-6239-4b65-86ad-b44efac9eeb5 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:27.646734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fafc63-561c-4dc7-8653-939751921ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:27.648068 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fafc63-561c-4dc7-8653-939751921ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:27.650272 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fafc63-561c-4dc7-8653-939751921ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:27.654858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fafc63-561c-4dc7-8653-939751921ddd admin admin] PATH_INFO: `/v3/projects/46162a8429564cfc9fd5ebda7f4df45f/users/46cc6cc894814466ab78ebf6da55ac2f/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:27.708738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fafc63-561c-4dc7-8653-939751921ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=46162a8429564cfc9fd5ebda7f4df45f, user_id=46cc6cc894814466ab78ebf6da55ac2f, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:27.712874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fafc63-561c-4dc7-8653-939751921ddd admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:27.805297 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 898/1779] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:27 2022] PUT /identity/v3/projects/46162a8429564cfc9fd5ebda7f4df45f/users/46cc6cc894814466ab78ebf6da55ac2f/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:27.807368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fafc63-561c-4dc7-8653-939751921ddd admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:27.814145 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e0fafc63-561c-4dc7-8653-939751921ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:27.815035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:27.817844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a3d2f478cf47db80e7e15b92b0fc99', 'request_id': 'req-d9136be1-c4ac-4af1-964f-9f2dfe22c136'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:27.932148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a3d2f478cf47db80e7e15b92b0fc99', 'request_id': 'req-d9136be1-c4ac-4af1-964f-9f2dfe22c136'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:27.949706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a3d2f478cf47db80e7e15b92b0fc99', 'request_id': 'req-d9136be1-c4ac-4af1-964f-9f2dfe22c136'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:27.975365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] The token cache is being invalidate because project 93a3d2f478cf47db80e7e15b92b0fc99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:27.976090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:27.977356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:27.979877 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 882/1780] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:27 2022] DELETE /identity/v3/projects/93a3d2f478cf47db80e7e15b92b0fc99 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:27.981597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:27.994632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:28.021701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fafc63-561c-4dc7-8653-939751921ddd admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:28.036642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129d114-d380-40a2-9a47-e09cb9072531 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:28.040269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129d114-d380-40a2-9a47-e09cb9072531 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:28.040744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129d114-d380-40a2-9a47-e09cb9072531 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:28.041083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129d114-d380-40a2-9a47-e09cb9072531 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:28.043144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5129d114-d380-40a2-9a47-e09cb9072531 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:28.064800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129d114-d380-40a2-9a47-e09cb9072531 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:28.072029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129d114-d380-40a2-9a47-e09cb9072531 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:28.085827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 899/1781] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:28.086881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129d114-d380-40a2-9a47-e09cb9072531 service nova] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:28.088329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5129d114-d380-40a2-9a47-e09cb9072531 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:28.175508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9136be1-c4ac-4af1-964f-9f2dfe22c136 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:28.224263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129d114-d380-40a2-9a47-e09cb9072531 service nova] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:28.234661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd6417b-25b0-443d-82cd-7157a101d9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:28.235569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6417b-25b0-443d-82cd-7157a101d9df admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:28.235917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6417b-25b0-443d-82cd-7157a101d9df admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:28.236195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd6417b-25b0-443d-82cd-7157a101d9df admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:28.237026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6417b-25b0-443d-82cd-7157a101d9df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:28.240358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd6417b-25b0-443d-82cd-7157a101d9df admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:28.258405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 900/1782] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:28.263612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479720fa-a6e6-4c29-bdc3-b69db3193489 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:28.263915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479720fa-a6e6-4c29-bdc3-b69db3193489 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:28.265854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479720fa-a6e6-4c29-bdc3-b69db3193489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:28.294022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-479720fa-a6e6-4c29-bdc3-b69db3193489 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:28.295370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-479720fa-a6e6-4c29-bdc3-b69db3193489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:28.314029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 901/1783] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:28.314029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479720fa-a6e6-4c29-bdc3-b69db3193489 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:28.321386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479720fa-a6e6-4c29-bdc3-b69db3193489 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:28.323941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a2919-2fdc-4c20-81ac-298ac184571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:28.324754 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2919-2fdc-4c20-81ac-298ac184571c admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:28.325152 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2919-2fdc-4c20-81ac-298ac184571c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:28.325561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a2919-2fdc-4c20-81ac-298ac184571c admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:28.326477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a2919-2fdc-4c20-81ac-298ac184571c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:28.329194 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a2919-2fdc-4c20-81ac-298ac184571c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:28.338678 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa547bc-e509-489b-afc5-78a6e52432d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:28.340168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa547bc-e509-489b-afc5-78a6e52432d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:28.340168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa547bc-e509-489b-afc5-78a6e52432d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:28.340168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa547bc-e509-489b-afc5-78a6e52432d9 admin admin] PATH_INFO: `/v3/projects/46162a8429564cfc9fd5ebda7f4df45f/users/46cc6cc894814466ab78ebf6da55ac2f/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:28.343101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa547bc-e509-489b-afc5-78a6e52432d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46162a8429564cfc9fd5ebda7f4df45f, user_id=46cc6cc894814466ab78ebf6da55ac2f, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:28.345883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa547bc-e509-489b-afc5-78a6e52432d9 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:28.365118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 902/1784] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:28 2022] PUT /identity/v3/projects/46162a8429564cfc9fd5ebda7f4df45f/users/46cc6cc894814466ab78ebf6da55ac2f/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:28.367113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b5422-e890-4642-9915-72909c123a60 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:28.367511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b5422-e890-4642-9915-72909c123a60 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:28.367851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b5422-e890-4642-9915-72909c123a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:28.402087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-981b5422-e890-4642-9915-72909c123a60 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:28.404009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-981b5422-e890-4642-9915-72909c123a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:28.436046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 883/1785] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:27 2022] POST /identity/v3/projects => generated 353 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:28.438005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ddb39-fbda-47c0-ba78-3d700e9610e3 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:28.439143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ddb39-fbda-47c0-ba78-3d700e9610e3 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:28.442655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ddb39-fbda-47c0-ba78-3d700e9610e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:28.488643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-8b0ddb39-fbda-47c0-ba78-3d700e9610e3 None None] MFA Rules not processed for user `46cc6cc894814466ab78ebf6da55ac2f`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:28.494082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8b0ddb39-fbda-47c0-ba78-3d700e9610e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:28.534266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 903/1786] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:28.535643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b5422-e890-4642-9915-72909c123a60 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:28.549510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b5422-e890-4642-9915-72909c123a60 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:28.639973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9be93-d41b-4320-93d0-c42876235c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:28.640934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9be93-d41b-4320-93d0-c42876235c31 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:28.641233 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9be93-d41b-4320-93d0-c42876235c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:28.641562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f9be93-d41b-4320-93d0-c42876235c31 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:28.642264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9be93-d41b-4320-93d0-c42876235c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:28.645663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f9be93-d41b-4320-93d0-c42876235c31 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:28.669179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 884/1787] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:28 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:28.670501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ddb39-fbda-47c0-ba78-3d700e9610e3 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:28.688593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:28.688786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ddb39-fbda-47c0-ba78-3d700e9610e3 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:28.693805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c56f-398d-4612-b61f-87749b4e4fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:28.694636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c56f-398d-4612-b61f-87749b4e4fae admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:28.695064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c56f-398d-4612-b61f-87749b4e4fae admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:28.695457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09c56f-398d-4612-b61f-87749b4e4fae admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:28.696292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09c56f-398d-4612-b61f-87749b4e4fae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:56:28.696716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09c56f-398d-4612-b61f-87749b4e4fae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:28.698884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09c56f-398d-4612-b61f-87749b4e4fae admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:28.716388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 885/1788] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:56:28.716388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c56f-398d-4612-b61f-87749b4e4fae admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:28.722511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ec09c56f-398d-4612-b61f-87749b4e4fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:28.749725 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 904/1789] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:28 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:28.750865 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9be93-d41b-4320-93d0-c42876235c31 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:28.763615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f9be93-d41b-4320-93d0-c42876235c31 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:28.771150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6a46c-fcb9-4227-b5c8-d1d5a2a5abdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:28.772242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a46c-fcb9-4227-b5c8-d1d5a2a5abdb admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:28.773362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a46c-fcb9-4227-b5c8-d1d5a2a5abdb admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:28.773787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6a46c-fcb9-4227-b5c8-d1d5a2a5abdb admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:28.780583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6a46c-fcb9-4227-b5c8-d1d5a2a5abdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:28.780583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6a46c-fcb9-4227-b5c8-d1d5a2a5abdb admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:28.852542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09c56f-398d-4612-b61f-87749b4e4fae admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:28.862562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b920a-477f-4c05-9137-12bd28312c21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:28.863239 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b920a-477f-4c05-9137-12bd28312c21 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:28.863567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b920a-477f-4c05-9137-12bd28312c21 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:28.863790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b920a-477f-4c05-9137-12bd28312c21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:28.875463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b920a-477f-4c05-9137-12bd28312c21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:28.876909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b920a-477f-4c05-9137-12bd28312c21 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:28.909546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 905/1790] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:28 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:28.910752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6a46c-fcb9-4227-b5c8-d1d5a2a5abdb admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:28.915508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 886/1791] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:28 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:28.916318 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b920a-477f-4c05-9137-12bd28312c21 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:28.938482 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b920a-477f-4c05-9137-12bd28312c21 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:29.091197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4ab3d-8c62-4b54-9508-df0ad456bd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:29.091964 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4ab3d-8c62-4b54-9508-df0ad456bd48 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:29.092347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4ab3d-8c62-4b54-9508-df0ad456bd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:29.092678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4ab3d-8c62-4b54-9508-df0ad456bd48 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:29.093701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4ab3d-8c62-4b54-9508-df0ad456bd48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:29.096936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4ab3d-8c62-4b54-9508-df0ad456bd48 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:29.109515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6a46c-fcb9-4227-b5c8-d1d5a2a5abdb admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:29.129864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe3b84-d3c3-490e-883b-b2bfee73fb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:29.129864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe3b84-d3c3-490e-883b-b2bfee73fb96 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:29.129864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe3b84-d3c3-490e-883b-b2bfee73fb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:29.130136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe3b84-d3c3-490e-883b-b2bfee73fb96 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:29.130880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe3b84-d3c3-490e-883b-b2bfee73fb96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:29.138230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe3b84-d3c3-490e-883b-b2bfee73fb96 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:29.158007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 906/1792] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:29.165603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe3b84-d3c3-490e-883b-b2bfee73fb96 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:29.201052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe3b84-d3c3-490e-883b-b2bfee73fb96 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:29.208270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdc318-a522-4829-a75c-5dcba65ccc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:29.209603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdc318-a522-4829-a75c-5dcba65ccc9a admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:29.210063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdc318-a522-4829-a75c-5dcba65ccc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:29.210521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdc318-a522-4829-a75c-5dcba65ccc9a admin admin] PATH_INFO: `/v3/projects/a8c4b821f8834a5b8e6d5f8b678d83ab/users/1800733a032d4f4fa773a787a4d7d91c/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:29.213075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 887/1793] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:28 2022] POST /identity/v3/users => generated 283 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:29.219043 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4ab3d-8c62-4b54-9508-df0ad456bd48 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:29.240493 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdc318-a522-4829-a75c-5dcba65ccc9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c4b821f8834a5b8e6d5f8b678d83ab, user_id=1800733a032d4f4fa773a787a4d7d91c, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:29.243658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdc318-a522-4829-a75c-5dcba65ccc9a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:29.246079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4ab3d-8c62-4b54-9508-df0ad456bd48 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:29.259102 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9310f-20b6-4611-845d-13103ff3e390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:29.260319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9310f-20b6-4611-845d-13103ff3e390 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:29.260781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9310f-20b6-4611-845d-13103ff3e390 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:29.261288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9310f-20b6-4611-845d-13103ff3e390 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:29.262198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9310f-20b6-4611-845d-13103ff3e390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:29.265028 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9310f-20b6-4611-845d-13103ff3e390 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:29.283051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 888/1794] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:29.284177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9310f-20b6-4611-845d-13103ff3e390 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:29.286592 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-bbc9310f-20b6-4611-845d-13103ff3e390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:29.345095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 907/1795] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:29 2022] PUT /identity/v3/projects/a8c4b821f8834a5b8e6d5f8b678d83ab/users/1800733a032d4f4fa773a787a4d7d91c/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:29.346345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdc318-a522-4829-a75c-5dcba65ccc9a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:29.453258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9310f-20b6-4611-845d-13103ff3e390 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:29.541866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdc318-a522-4829-a75c-5dcba65ccc9a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:29.542042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b10ae1-30df-4c9e-8c21-1ddecd614e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:29.542645 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b10ae1-30df-4c9e-8c21-1ddecd614e12 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:29.542930 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b10ae1-30df-4c9e-8c21-1ddecd614e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:29.543205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b10ae1-30df-4c9e-8c21-1ddecd614e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:29.547795 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-97b10ae1-30df-4c9e-8c21-1ddecd614e12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:29.553502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fadb6-7294-4536-9f56-c8faf39e227d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:29.553502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fadb6-7294-4536-9f56-c8faf39e227d admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:29.556191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fadb6-7294-4536-9f56-c8faf39e227d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:29.556191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fadb6-7294-4536-9f56-c8faf39e227d admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:29.556191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fadb6-7294-4536-9f56-c8faf39e227d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:29.557584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059fadb6-7294-4536-9f56-c8faf39e227d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:29.581629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 908/1796] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:29.583043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fadb6-7294-4536-9f56-c8faf39e227d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:29.602255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fadb6-7294-4536-9f56-c8faf39e227d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:29.608768 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c7337-2ccf-426c-8f44-43ca4427823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:29.609474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c7337-2ccf-426c-8f44-43ca4427823c admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:29.609760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c7337-2ccf-426c-8f44-43ca4427823c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:29.614624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c7337-2ccf-426c-8f44-43ca4427823c admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:29.615653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c7337-2ccf-426c-8f44-43ca4427823c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:29.617507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1c7337-2ccf-426c-8f44-43ca4427823c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:29.631999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 909/1797] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:29.631999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c7337-2ccf-426c-8f44-43ca4427823c admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:29.636423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:29.643359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1c7337-2ccf-426c-8f44-43ca4427823c admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:29.649600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deede19-d90b-42cc-9b3f-668e63614645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:29.650504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deede19-d90b-42cc-9b3f-668e63614645 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:29.650792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deede19-d90b-42cc-9b3f-668e63614645 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:29.651076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deede19-d90b-42cc-9b3f-668e63614645 admin admin] PATH_INFO: `/v3/projects/d475b60042f24164a36ae00d6b137760/users/55255f04f5d0401b96b3838f388529d3/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:29.688370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deede19-d90b-42cc-9b3f-668e63614645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d475b60042f24164a36ae00d6b137760, user_id=55255f04f5d0401b96b3838f388529d3, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:29.693309 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deede19-d90b-42cc-9b3f-668e63614645 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:29.707028 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b10ae1-30df-4c9e-8c21-1ddecd614e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:29.709290 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b10ae1-30df-4c9e-8c21-1ddecd614e12 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:29.720680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 889/1798] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:29.721953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b10ae1-30df-4c9e-8c21-1ddecd614e12 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:29.736174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b10ae1-30df-4c9e-8c21-1ddecd614e12 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:29.749886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c760f-d794-41f2-bcbe-31b2c97f498a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:29.751435 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c760f-d794-41f2-bcbe-31b2c97f498a admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:29.751929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c760f-d794-41f2-bcbe-31b2c97f498a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:29.753468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c760f-d794-41f2-bcbe-31b2c97f498a admin admin] PATH_INFO: `/v3/projects/a8c4b821f8834a5b8e6d5f8b678d83ab/users/1800733a032d4f4fa773a787a4d7d91c/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:29.759505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c760f-d794-41f2-bcbe-31b2c97f498a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c4b821f8834a5b8e6d5f8b678d83ab, user_id=1800733a032d4f4fa773a787a4d7d91c, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:29.766450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c760f-d794-41f2-bcbe-31b2c97f498a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:29.799968 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 890/1799] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:29 2022] PUT /identity/v3/projects/a8c4b821f8834a5b8e6d5f8b678d83ab/users/1800733a032d4f4fa773a787a4d7d91c/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:29.804155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e5e7b-6181-45fb-b1c6-c668263c2bc0 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:29.804448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e5e7b-6181-45fb-b1c6-c668263c2bc0 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:29.804727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e5e7b-6181-45fb-b1c6-c668263c2bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:29.805622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 910/1800] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:29 2022] PUT /identity/v3/projects/d475b60042f24164a36ae00d6b137760/users/55255f04f5d0401b96b3838f388529d3/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:29.816723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4956845-4981-440f-acc0-12801a47098e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:29.820020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4956845-4981-440f-acc0-12801a47098e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:29.821503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4956845-4981-440f-acc0-12801a47098e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:29.837255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-450e5e7b-6181-45fb-b1c6-c668263c2bc0 None None] MFA Rules not processed for user `1800733a032d4f4fa773a787a4d7d91c`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:29.838529 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-450e5e7b-6181-45fb-b1c6-c668263c2bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:29.864450 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f4956845-4981-440f-acc0-12801a47098e None None] MFA Rules not processed for user `55255f04f5d0401b96b3838f388529d3`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:29.866013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f4956845-4981-440f-acc0-12801a47098e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:29.929160 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:29.959855 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 891/1801] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:29 2022] POST /identity/v3/auth/tokens => generated 3287 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:29.976285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e5e7b-6181-45fb-b1c6-c668263c2bc0 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:30.047141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 911/1802] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:30.049486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4956845-4981-440f-acc0-12801a47098e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:30.141476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e5e7b-6181-45fb-b1c6-c668263c2bc0 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:30.149022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cadd81-af4e-4630-9397-820fa3325946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:30.150371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cadd81-af4e-4630-9397-820fa3325946 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:30.150993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cadd81-af4e-4630-9397-820fa3325946 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:30.151379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cadd81-af4e-4630-9397-820fa3325946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:30.209921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cadd81-af4e-4630-9397-820fa3325946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:30.211357 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4956845-4981-440f-acc0-12801a47098e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:30.212136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cadd81-af4e-4630-9397-820fa3325946 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:30.226990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00889a56-bbc7-4ca3-87df-09a6323099f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:30.227788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00889a56-bbc7-4ca3-87df-09a6323099f7 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:30.228176 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00889a56-bbc7-4ca3-87df-09a6323099f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:30.228567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00889a56-bbc7-4ca3-87df-09a6323099f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:30.229893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00889a56-bbc7-4ca3-87df-09a6323099f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:30.233699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00889a56-bbc7-4ca3-87df-09a6323099f7 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:30.256983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 892/1803] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:29 2022] GET /identity/v3/auth/tokens => generated 3287 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:30.414564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 912/1804] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:30 2022] POST /identity/v3/projects => generated 365 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:30.415495 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cadd81-af4e-4630-9397-820fa3325946 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:30.429511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cadd81-af4e-4630-9397-820fa3325946 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:30.529273 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cef6be-6683-40b8-9bfd-d2086909d1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:30.530545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cef6be-6683-40b8-9bfd-d2086909d1bf admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:30.531037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cef6be-6683-40b8-9bfd-d2086909d1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:30.531310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cef6be-6683-40b8-9bfd-d2086909d1bf admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:30.533772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cef6be-6683-40b8-9bfd-d2086909d1bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:30.537207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cef6be-6683-40b8-9bfd-d2086909d1bf admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:30.573150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00889a56-bbc7-4ca3-87df-09a6323099f7 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:30.652734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 893/1805] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:30 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:30.657322 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cef6be-6683-40b8-9bfd-d2086909d1bf admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:30.660330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00889a56-bbc7-4ca3-87df-09a6323099f7 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:30.667207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce35b8-2e25-4782-83a8-bad28405ad5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:30.668029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce35b8-2e25-4782-83a8-bad28405ad5f service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:30.668704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce35b8-2e25-4782-83a8-bad28405ad5f service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:30.668999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce35b8-2e25-4782-83a8-bad28405ad5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:30.671835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b5ce35b8-2e25-4782-83a8-bad28405ad5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:30.672678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cef6be-6683-40b8-9bfd-d2086909d1bf admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:30.678520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85002ed9-9933-46d9-9b4d-754d6c679269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:30.679321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85002ed9-9933-46d9-9b4d-754d6c679269 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:30.679715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85002ed9-9933-46d9-9b4d-754d6c679269 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:30.683955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85002ed9-9933-46d9-9b4d-754d6c679269 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:30.683955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85002ed9-9933-46d9-9b4d-754d6c679269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:30.683955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85002ed9-9933-46d9-9b4d-754d6c679269 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:30.699789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 894/1806] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:30.701198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85002ed9-9933-46d9-9b4d-754d6c679269 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:30.733910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85002ed9-9933-46d9-9b4d-754d6c679269 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:30.749341 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42ce15-270c-40c8-b546-f0e4b7c7acc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:30.750419 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42ce15-270c-40c8-b546-f0e4b7c7acc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:30.750803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42ce15-270c-40c8-b546-f0e4b7c7acc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:30.751181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42ce15-270c-40c8-b546-f0e4b7c7acc4 admin admin] PATH_INFO: `/v3/projects/b6cd7ce980fe48fbafd8aea4b8bf9ce8/users/a35619a6539a4b6e92b2f820af45dc5c/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:30.797118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42ce15-270c-40c8-b546-f0e4b7c7acc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6cd7ce980fe48fbafd8aea4b8bf9ce8, user_id=a35619a6539a4b6e92b2f820af45dc5c, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:30.800730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42ce15-270c-40c8-b546-f0e4b7c7acc4 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:30.827231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce35b8-2e25-4782-83a8-bad28405ad5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:30.828693 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce35b8-2e25-4782-83a8-bad28405ad5f service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:30.840739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 913/1807] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:30 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:56:30.902916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 895/1808] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:30 2022] PUT /identity/v3/projects/b6cd7ce980fe48fbafd8aea4b8bf9ce8/users/a35619a6539a4b6e92b2f820af45dc5c/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:30.904419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce35b8-2e25-4782-83a8-bad28405ad5f service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:30.920880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce35b8-2e25-4782-83a8-bad28405ad5f service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:30.963396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:31.019588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eed4283-e19f-488b-a2d6-cc83fed3fa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:31.020435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed4283-e19f-488b-a2d6-cc83fed3fa15 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:31.020854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed4283-e19f-488b-a2d6-cc83fed3fa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:31.021323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eed4283-e19f-488b-a2d6-cc83fed3fa15 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:31.022265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed4283-e19f-488b-a2d6-cc83fed3fa15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:31.024636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eed4283-e19f-488b-a2d6-cc83fed3fa15 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:31.039895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 914/1809] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:31.043870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42ce15-270c-40c8-b546-f0e4b7c7acc4 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:31.059980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42ce15-270c-40c8-b546-f0e4b7c7acc4 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:31.065448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491c903-7b0c-4cc4-82d9-8d1e50b4bebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:31.066348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491c903-7b0c-4cc4-82d9-8d1e50b4bebd admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:31.071337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491c903-7b0c-4cc4-82d9-8d1e50b4bebd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:31.073076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491c903-7b0c-4cc4-82d9-8d1e50b4bebd admin admin] PATH_INFO: `/v3/projects/b6cd7ce980fe48fbafd8aea4b8bf9ce8/users/a35619a6539a4b6e92b2f820af45dc5c/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:31.075524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491c903-7b0c-4cc4-82d9-8d1e50b4bebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6cd7ce980fe48fbafd8aea4b8bf9ce8, user_id=a35619a6539a4b6e92b2f820af45dc5c, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:31.090571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491c903-7b0c-4cc4-82d9-8d1e50b4bebd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:31.117614 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 896/1810] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:31 2022] PUT /identity/v3/projects/b6cd7ce980fe48fbafd8aea4b8bf9ce8/users/a35619a6539a4b6e92b2f820af45dc5c/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:56:31.118763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ccc2-2217-490c-89b6-b8bbaecfad9d None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:31.119057 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ccc2-2217-490c-89b6-b8bbaecfad9d None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:31.119345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62ccc2-2217-490c-89b6-b8bbaecfad9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:31.160318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6c62ccc2-2217-490c-89b6-b8bbaecfad9d None None] MFA Rules not processed for user `a35619a6539a4b6e92b2f820af45dc5c`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:31.162293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6c62ccc2-2217-490c-89b6-b8bbaecfad9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:31.251490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:31.261484 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 915/1811] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:31.319938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491c903-7b0c-4cc4-82d9-8d1e50b4bebd admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:31.392306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491c903-7b0c-4cc4-82d9-8d1e50b4bebd admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:31.392306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3437152-206e-4abd-90b4-ba4e93379cd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:31.392306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3437152-206e-4abd-90b4-ba4e93379cd0 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:31.392306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3437152-206e-4abd-90b4-ba4e93379cd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:31.392635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3437152-206e-4abd-90b4-ba4e93379cd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:31.400414 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3437152-206e-4abd-90b4-ba4e93379cd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:31.402082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3437152-206e-4abd-90b4-ba4e93379cd0 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:31.414636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 897/1812] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:56:31.826613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62ccc2-2217-490c-89b6-b8bbaecfad9d None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:31.836680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62ccc2-2217-490c-89b6-b8bbaecfad9d None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:31.840556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bda84-75dd-4101-80bc-04caf8457589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:31.843173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bda84-75dd-4101-80bc-04caf8457589 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:31.843672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bda84-75dd-4101-80bc-04caf8457589 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:31.843884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bda84-75dd-4101-80bc-04caf8457589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:31.910309 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bda84-75dd-4101-80bc-04caf8457589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:31.911621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bda84-75dd-4101-80bc-04caf8457589 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:31.925528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 916/1813] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:32.056680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:32.266001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:33.000899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:33.993090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3437152-206e-4abd-90b4-ba4e93379cd0 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:34.027460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:34.055907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3437152-206e-4abd-90b4-ba4e93379cd0 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:34.060337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:34.061772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:34.062061 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:34.062333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] PATH_INFO: `/v3/users/3bbf01dcd9b748eb9e8f452b8aa28d2c` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:34.063516 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bbf01dcd9b748eb9e8f452b8aa28d2c)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:34.065454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:34.320675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bbf01dcd9b748eb9e8f452b8aa28d2c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:34.329625 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bbf01dcd9b748eb9e8f452b8aa28d2c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:34.330798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bbf01dcd9b748eb9e8f452b8aa28d2c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:34.346984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bbf01dcd9b748eb9e8f452b8aa28d2c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:34.348325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bbf01dcd9b748eb9e8f452b8aa28d2c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:34.442146 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 898/1814] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:33 2022] DELETE /identity/v3/users/3bbf01dcd9b748eb9e8f452b8aa28d2c => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:34.444851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f789717-18a8-4f2f-a85b-8750f26dce4e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:34.449077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f789717-18a8-4f2f-a85b-8750f26dce4e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:34.449077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f789717-18a8-4f2f-a85b-8750f26dce4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:34.467950 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3f789717-18a8-4f2f-a85b-8750f26dce4e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:34.469148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3f789717-18a8-4f2f-a85b-8750f26dce4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:34.546880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 917/1815] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:34.555751 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:34.558243 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:34.632266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a9fd9-1ae0-4eb0-9453-e5928b66c585 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:34.636085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce4688-1aad-4fad-98ca-b3a376c6b38d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:34.636839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce4688-1aad-4fad-98ca-b3a376c6b38d service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:34.639537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce4688-1aad-4fad-98ca-b3a376c6b38d service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:34.639537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce4688-1aad-4fad-98ca-b3a376c6b38d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:34.647086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce4688-1aad-4fad-98ca-b3a376c6b38d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:34.648531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce4688-1aad-4fad-98ca-b3a376c6b38d service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:34.659714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 899/1816] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:56:35.223948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f789717-18a8-4f2f-a85b-8750f26dce4e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:35.225430 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3f789717-18a8-4f2f-a85b-8750f26dce4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:35.239574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f789717-18a8-4f2f-a85b-8750f26dce4e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:35.250200 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:35.250200 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:35.250200 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:35.250200 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] PATH_INFO: `/v3/projects/37258dbf12af4505bbe44a542a261a61` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:35.250828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] RBAC: Authorizing `identity:delete_project(project_id=37258dbf12af4505bbe44a542a261a61)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:35.251193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:35.391060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37258dbf12af4505bbe44a542a261a61', 'request_id': 'req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:35.486996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37258dbf12af4505bbe44a542a261a61', 'request_id': 'req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:35.494737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37258dbf12af4505bbe44a542a261a61', 'request_id': 'req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:35.519755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] The token cache is being invalidate because project 37258dbf12af4505bbe44a542a261a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:35.520063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:35.520748 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:35.526000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 918/1817] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:35 2022] DELETE /identity/v3/projects/37258dbf12af4505bbe44a542a261a61 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:35.527161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce4688-1aad-4fad-98ca-b3a376c6b38d service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:35.541149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce4688-1aad-4fad-98ca-b3a376c6b38d service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:35.601102 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:35.625890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb283196-7df3-4f93-9845-db6d5e9aa867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:35.627121 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb283196-7df3-4f93-9845-db6d5e9aa867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:35.627746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb283196-7df3-4f93-9845-db6d5e9aa867 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:35.628253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb283196-7df3-4f93-9845-db6d5e9aa867 admin admin] PATH_INFO: `/v3/users/46cc6cc894814466ab78ebf6da55ac2f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:35.630628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb283196-7df3-4f93-9845-db6d5e9aa867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46cc6cc894814466ab78ebf6da55ac2f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:35.633402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb283196-7df3-4f93-9845-db6d5e9aa867 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:35.932212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-cb283196-7df3-4f93-9845-db6d5e9aa867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46cc6cc894814466ab78ebf6da55ac2f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:35.951330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-cb283196-7df3-4f93-9845-db6d5e9aa867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46cc6cc894814466ab78ebf6da55ac2f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:35.952104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-cb283196-7df3-4f93-9845-db6d5e9aa867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46cc6cc894814466ab78ebf6da55ac2f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:35.969029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-cb283196-7df3-4f93-9845-db6d5e9aa867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46cc6cc894814466ab78ebf6da55ac2f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:35.970506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-cb283196-7df3-4f93-9845-db6d5e9aa867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46cc6cc894814466ab78ebf6da55ac2f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:36.084761 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 900/1818] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:35 2022] DELETE /identity/v3/users/46cc6cc894814466ab78ebf6da55ac2f => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:36.606439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:36.608071 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:36.651661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea9731-7678-45d5-be9c-fe9c0ffe194c admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:36.729818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:36.730705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:36.731249 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:36.731528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] PATH_INFO: `/v3/projects/46162a8429564cfc9fd5ebda7f4df45f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:36.732837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46162a8429564cfc9fd5ebda7f4df45f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:36.734760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:36.743069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:37.590385 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46162a8429564cfc9fd5ebda7f4df45f', 'request_id': 'req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:37.667519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46162a8429564cfc9fd5ebda7f4df45f', 'request_id': 'req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:37.675011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46162a8429564cfc9fd5ebda7f4df45f', 'request_id': 'req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:37.688138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] The token cache is being invalidate because project 46162a8429564cfc9fd5ebda7f4df45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:37.688832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:37.689584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-87b18f90-5fb1-4f9b-bba9-d7cdfed2e8d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:37.691472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 919/1819] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:36 2022] DELETE /identity/v3/projects/46162a8429564cfc9fd5ebda7f4df45f => generated 0 bytes in 1086 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:37.696116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13384d-1f64-4f51-b044-6fbaa4304d65 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:37.696693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13384d-1f64-4f51-b044-6fbaa4304d65 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:37.696693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13384d-1f64-4f51-b044-6fbaa4304d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:37.721824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-7e13384d-1f64-4f51-b044-6fbaa4304d65 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:37.723048 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-7e13384d-1f64-4f51-b044-6fbaa4304d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:37.796717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 901/1820] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:37.804445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9f628-bb2d-4d21-aa20-f8f280b40160 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:37.804805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9f628-bb2d-4d21-aa20-f8f280b40160 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:37.805161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9f628-bb2d-4d21-aa20-f8f280b40160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:37.823966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9ff9f628-bb2d-4d21-aa20-f8f280b40160 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:37.825100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9ff9f628-bb2d-4d21-aa20-f8f280b40160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:37.833817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 920/1821] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:37.837952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13384d-1f64-4f51-b044-6fbaa4304d65 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:37.847598 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13384d-1f64-4f51-b044-6fbaa4304d65 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:37.852251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdfb96-f28f-4360-a57c-771327184ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:37.852912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdfb96-f28f-4360-a57c-771327184ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:37.853198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdfb96-f28f-4360-a57c-771327184ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:37.853508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdfb96-f28f-4360-a57c-771327184ae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:37.854226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdfb96-f28f-4360-a57c-771327184ae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:56:37.854538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdfb96-f28f-4360-a57c-771327184ae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:37.856409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdfb96-f28f-4360-a57c-771327184ae2 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:37.868740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 902/1822] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:56:37.874353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9f628-bb2d-4d21-aa20-f8f280b40160 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:37.884166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9f628-bb2d-4d21-aa20-f8f280b40160 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:37.888940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4891fe-62c6-4206-b22a-f0a05f9d297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:37.889804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4891fe-62c6-4206-b22a-f0a05f9d297d admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:37.890091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4891fe-62c6-4206-b22a-f0a05f9d297d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:37.890364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4891fe-62c6-4206-b22a-f0a05f9d297d admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:37.891048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4891fe-62c6-4206-b22a-f0a05f9d297d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:37.893080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4891fe-62c6-4206-b22a-f0a05f9d297d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:37.954302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 921/1823] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:37 2022] POST /identity/v3/projects => generated 389 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:37.958443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdfb96-f28f-4360-a57c-771327184ae2 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:37.968267 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdfb96-f28f-4360-a57c-771327184ae2 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:38.024400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27686b-c7b3-4c18-accc-88f6c6af65c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:38.025086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27686b-c7b3-4c18-accc-88f6c6af65c1 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:38.025406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27686b-c7b3-4c18-accc-88f6c6af65c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:38.025689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd27686b-c7b3-4c18-accc-88f6c6af65c1 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:38.026470 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27686b-c7b3-4c18-accc-88f6c6af65c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:38.028405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd27686b-c7b3-4c18-accc-88f6c6af65c1 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:38.035122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:38.125298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 903/1824] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:37 2022] POST /identity/v3/users => generated 295 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:38.129679 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4891fe-62c6-4206-b22a-f0a05f9d297d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:38.142279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4891fe-62c6-4206-b22a-f0a05f9d297d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:38.147256 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6eafc-4bb2-4c4e-b354-ca29ebf36ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:38.149566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6eafc-4bb2-4c4e-b354-ca29ebf36ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:38.149566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6eafc-4bb2-4c4e-b354-ca29ebf36ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:38.149566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6eafc-4bb2-4c4e-b354-ca29ebf36ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:38.149566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6eafc-4bb2-4c4e-b354-ca29ebf36ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:38.150941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6eafc-4bb2-4c4e-b354-ca29ebf36ec5 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:38.162060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 922/1825] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:38.166222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27686b-c7b3-4c18-accc-88f6c6af65c1 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:38.176764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd27686b-c7b3-4c18-accc-88f6c6af65c1 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:38.181611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5783863-95e3-47e4-a3e5-d9e2608d6671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:38.182264 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5783863-95e3-47e4-a3e5-d9e2608d6671 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:38.182549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5783863-95e3-47e4-a3e5-d9e2608d6671 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:38.182829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5783863-95e3-47e4-a3e5-d9e2608d6671 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:38.183539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5783863-95e3-47e4-a3e5-d9e2608d6671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:38.185161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5783863-95e3-47e4-a3e5-d9e2608d6671 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:38.196531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 904/1826] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:38.200444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6eafc-4bb2-4c4e-b354-ca29ebf36ec5 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:38.210244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6eafc-4bb2-4c4e-b354-ca29ebf36ec5 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:38.214877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d0763-e927-4412-a08d-940a7807283c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:38.215740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0763-e927-4412-a08d-940a7807283c admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:38.216027 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0763-e927-4412-a08d-940a7807283c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:38.216304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d0763-e927-4412-a08d-940a7807283c admin admin] PATH_INFO: `/v3/projects/757411161139490196452f431915b2ba/users/9d46a016eac8442eafc04708a31ebf84/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:38.230876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d0763-e927-4412-a08d-940a7807283c admin admin] RBAC: Authorizing `identity:create_grant(project_id=757411161139490196452f431915b2ba, user_id=9d46a016eac8442eafc04708a31ebf84, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:38.234456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d0763-e927-4412-a08d-940a7807283c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:38.307949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 923/1827] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:38 2022] PUT /identity/v3/projects/757411161139490196452f431915b2ba/users/9d46a016eac8442eafc04708a31ebf84/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:38.319396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1ec3b-91af-4b04-81e4-7350f89cac21 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:38.319847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1ec3b-91af-4b04-81e4-7350f89cac21 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:38.320177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af1ec3b-91af-4b04-81e4-7350f89cac21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:38.351750 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-9af1ec3b-91af-4b04-81e4-7350f89cac21 None None] MFA Rules not processed for user `9d46a016eac8442eafc04708a31ebf84`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:38.353068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-9af1ec3b-91af-4b04-81e4-7350f89cac21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:38.422285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 905/1828] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:38.426209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d0763-e927-4412-a08d-940a7807283c admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:38.494204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d0763-e927-4412-a08d-940a7807283c admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:38.499494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721154e-b013-4e33-b0d8-2d9a00e57886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:38.500222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721154e-b013-4e33-b0d8-2d9a00e57886 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:38.500580 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721154e-b013-4e33-b0d8-2d9a00e57886 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:38.500914 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721154e-b013-4e33-b0d8-2d9a00e57886 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:38.501850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721154e-b013-4e33-b0d8-2d9a00e57886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:38.503977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721154e-b013-4e33-b0d8-2d9a00e57886 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:38.577050 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 924/1829] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:38 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:38.581006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1ec3b-91af-4b04-81e4-7350f89cac21 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:38.591728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af1ec3b-91af-4b04-81e4-7350f89cac21 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:38.646384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083513e7-93b6-4255-ac86-3e30f68ce61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:38.647049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083513e7-93b6-4255-ac86-3e30f68ce61f admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:38.647335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083513e7-93b6-4255-ac86-3e30f68ce61f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:38.647636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083513e7-93b6-4255-ac86-3e30f68ce61f admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:38.648363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083513e7-93b6-4255-ac86-3e30f68ce61f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:38.650312 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083513e7-93b6-4255-ac86-3e30f68ce61f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:38.723188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 906/1830] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:38 2022] POST /identity/v3/users => generated 300 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:38.727127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721154e-b013-4e33-b0d8-2d9a00e57886 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:38.738586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721154e-b013-4e33-b0d8-2d9a00e57886 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:38.743529 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d647c62-f69d-4ea9-898a-be369fea9b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:38.744373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d647c62-f69d-4ea9-898a-be369fea9b83 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:38.744808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d647c62-f69d-4ea9-898a-be369fea9b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:38.745211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d647c62-f69d-4ea9-898a-be369fea9b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:38.746285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d647c62-f69d-4ea9-898a-be369fea9b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:38.747777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d647c62-f69d-4ea9-898a-be369fea9b83 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:38.762641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 925/1831] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:38.766921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083513e7-93b6-4255-ac86-3e30f68ce61f admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:38.779577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083513e7-93b6-4255-ac86-3e30f68ce61f admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:38.784316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b03f2-dd79-4b24-a07d-6d65e5f820bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:38.789573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b03f2-dd79-4b24-a07d-6d65e5f820bc admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:38.789887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b03f2-dd79-4b24-a07d-6d65e5f820bc admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:38.790177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b03f2-dd79-4b24-a07d-6d65e5f820bc admin admin] PATH_INFO: `/v3/projects/6638a8585f334de08104980a8e83536a/users/436eb6206d62480a9f5f642faa6051e5/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:38.807271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b03f2-dd79-4b24-a07d-6d65e5f820bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6638a8585f334de08104980a8e83536a, user_id=436eb6206d62480a9f5f642faa6051e5, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:38.810088 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b03f2-dd79-4b24-a07d-6d65e5f820bc admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:38.879652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 907/1832] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:38 2022] PUT /identity/v3/projects/6638a8585f334de08104980a8e83536a/users/436eb6206d62480a9f5f642faa6051e5/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:38.883549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d647c62-f69d-4ea9-898a-be369fea9b83 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:38.893921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d647c62-f69d-4ea9-898a-be369fea9b83 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:38.897787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:38.954767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2e166-4195-410a-86f3-7ff3a9f2936e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:38.955361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2e166-4195-410a-86f3-7ff3a9f2936e admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:38.955648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2e166-4195-410a-86f3-7ff3a9f2936e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:38.955926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2e166-4195-410a-86f3-7ff3a9f2936e admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:38.956817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2e166-4195-410a-86f3-7ff3a9f2936e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:38.958361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2e166-4195-410a-86f3-7ff3a9f2936e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:38.969383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 926/1833] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:38.977777 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b03f2-dd79-4b24-a07d-6d65e5f820bc admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:38.989082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b03f2-dd79-4b24-a07d-6d65e5f820bc admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:38.994006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e76ddf-d5ce-4b53-bc08-b6f9e5d31efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:38.994864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e76ddf-d5ce-4b53-bc08-b6f9e5d31efc admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:38.995150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e76ddf-d5ce-4b53-bc08-b6f9e5d31efc admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:38.995455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e76ddf-d5ce-4b53-bc08-b6f9e5d31efc admin admin] PATH_INFO: `/v3/projects/6638a8585f334de08104980a8e83536a/users/436eb6206d62480a9f5f642faa6051e5/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:38.997483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e76ddf-d5ce-4b53-bc08-b6f9e5d31efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6638a8585f334de08104980a8e83536a, user_id=436eb6206d62480a9f5f642faa6051e5, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:39.000902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e76ddf-d5ce-4b53-bc08-b6f9e5d31efc admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:39.011221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 908/1834] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:38 2022] PUT /identity/v3/projects/6638a8585f334de08104980a8e83536a/users/436eb6206d62480a9f5f642faa6051e5/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:39.021397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72621367-44ef-4cbc-a40e-662982ce5323 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:39.021767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72621367-44ef-4cbc-a40e-662982ce5323 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:39.022052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72621367-44ef-4cbc-a40e-662982ce5323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:39.049039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-72621367-44ef-4cbc-a40e-662982ce5323 None None] MFA Rules not processed for user `436eb6206d62480a9f5f642faa6051e5`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:39.050275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-72621367-44ef-4cbc-a40e-662982ce5323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:39.119997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 927/1835] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:39.130724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e76ddf-d5ce-4b53-bc08-b6f9e5d31efc admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:39.132444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-07e76ddf-d5ce-4b53-bc08-b6f9e5d31efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:39.156886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:39.191148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e76ddf-d5ce-4b53-bc08-b6f9e5d31efc admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:39.194870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659a933-6a58-4b2e-acdf-c19ac80c6083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:39.195509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659a933-6a58-4b2e-acdf-c19ac80c6083 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:39.195788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659a933-6a58-4b2e-acdf-c19ac80c6083 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:39.196059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659a933-6a58-4b2e-acdf-c19ac80c6083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:39.205246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659a933-6a58-4b2e-acdf-c19ac80c6083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:39.206688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659a933-6a58-4b2e-acdf-c19ac80c6083 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:39.215830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 909/1836] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:39.339102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72621367-44ef-4cbc-a40e-662982ce5323 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:39.400685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72621367-44ef-4cbc-a40e-662982ce5323 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:39.405100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:39.405983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:39.406260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:39.406527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] PATH_INFO: `/v3/users/9d46a016eac8442eafc04708a31ebf84` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:39.407701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d46a016eac8442eafc04708a31ebf84)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:39.409637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:39.576973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:39.590199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d46a016eac8442eafc04708a31ebf84'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:39.590976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d46a016eac8442eafc04708a31ebf84'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:39.598131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d46a016eac8442eafc04708a31ebf84'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:39.598880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d46a016eac8442eafc04708a31ebf84'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:39.612726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d46a016eac8442eafc04708a31ebf84'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:39.684286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 928/1837] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:39 2022] DELETE /identity/v3/users/9d46a016eac8442eafc04708a31ebf84 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:39.689230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbabcf-ed44-4bd0-b8a1-4d11a520b572 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:39.689554 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbabcf-ed44-4bd0-b8a1-4d11a520b572 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:39.689962 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bbabcf-ed44-4bd0-b8a1-4d11a520b572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:39.709393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-06bbabcf-ed44-4bd0-b8a1-4d11a520b572 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:39.710689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-06bbabcf-ed44-4bd0-b8a1-4d11a520b572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:39.769825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 910/1838] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:39.779399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:39.780851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:39.836966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706fd73e-6e58-42b4-92c0-c007f21ce586 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:39.840612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c8fc8-07e5-47e0-9400-23000702a2da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:39.841282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c8fc8-07e5-47e0-9400-23000702a2da service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:39.841597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c8fc8-07e5-47e0-9400-23000702a2da service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:39.841871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c8fc8-07e5-47e0-9400-23000702a2da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:39.851129 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c8fc8-07e5-47e0-9400-23000702a2da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:39.852550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c8fc8-07e5-47e0-9400-23000702a2da service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:39.862314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 929/1839] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:40.314247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbabcf-ed44-4bd0-b8a1-4d11a520b572 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:40.315650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-06bbabcf-ed44-4bd0-b8a1-4d11a520b572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:40.327420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bbabcf-ed44-4bd0-b8a1-4d11a520b572 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:40.333699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:40.335211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:40.335502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:40.335961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] PATH_INFO: `/v3/projects/757411161139490196452f431915b2ba` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:40.337216 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=757411161139490196452f431915b2ba)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:40.339290 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:40.477953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '757411161139490196452f431915b2ba', 'request_id': 'req-4a01447b-7ba0-476e-ac13-890c49d34a39'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:40.558117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '757411161139490196452f431915b2ba', 'request_id': 'req-4a01447b-7ba0-476e-ac13-890c49d34a39'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:40.563785 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '757411161139490196452f431915b2ba', 'request_id': 'req-4a01447b-7ba0-476e-ac13-890c49d34a39'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:40.574640 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] The token cache is being invalidate because project 757411161139490196452f431915b2ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:40.574927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:40.575558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:40.579903 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 911/1840] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:40 2022] DELETE /identity/v3/projects/757411161139490196452f431915b2ba => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:40.581111 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c8fc8-07e5-47e0-9400-23000702a2da service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:40.590950 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c8fc8-07e5-47e0-9400-23000702a2da service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:40.647587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6aac00-b3df-487e-a4e5-0d0988243ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:40.648379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6aac00-b3df-487e-a4e5-0d0988243ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:40.648665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6aac00-b3df-487e-a4e5-0d0988243ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:40.648944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6aac00-b3df-487e-a4e5-0d0988243ba8 admin admin] PATH_INFO: `/v3/users/436eb6206d62480a9f5f642faa6051e5` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:40.650165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6aac00-b3df-487e-a4e5-0d0988243ba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=436eb6206d62480a9f5f642faa6051e5)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:40.652082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6aac00-b3df-487e-a4e5-0d0988243ba8 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:40.807376 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6d6aac00-b3df-487e-a4e5-0d0988243ba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '436eb6206d62480a9f5f642faa6051e5'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:40.808043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6d6aac00-b3df-487e-a4e5-0d0988243ba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '436eb6206d62480a9f5f642faa6051e5'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:40.812504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:40.815361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6d6aac00-b3df-487e-a4e5-0d0988243ba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '436eb6206d62480a9f5f642faa6051e5'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:40.816012 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6d6aac00-b3df-487e-a4e5-0d0988243ba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '436eb6206d62480a9f5f642faa6051e5'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:40.828273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6d6aac00-b3df-487e-a4e5-0d0988243ba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '436eb6206d62480a9f5f642faa6051e5'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:40.908402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 930/1841] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:40 2022] DELETE /identity/v3/users/436eb6206d62480a9f5f642faa6051e5 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:41.304257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:41.306367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:41.320335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01447b-7ba0-476e-ac13-890c49d34a39 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:41.401704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:41.416280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:41.417195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:41.417516 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:41.417832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] PATH_INFO: `/v3/projects/6638a8585f334de08104980a8e83536a` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:41.419054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6638a8585f334de08104980a8e83536a)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:41.423864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:41.554421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6638a8585f334de08104980a8e83536a', 'request_id': 'req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:41.611953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6638a8585f334de08104980a8e83536a', 'request_id': 'req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:41.618863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6638a8585f334de08104980a8e83536a', 'request_id': 'req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:41.635032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] The token cache is being invalidate because project 6638a8585f334de08104980a8e83536a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:41.635427 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:41.636814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ca298a34-f3e3-4c87-a01b-bcf2558fc8f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:41.642090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 912/1842] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:41 2022] DELETE /identity/v3/projects/6638a8585f334de08104980a8e83536a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:41.643817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b587b-5fca-4aad-9b4f-7e3baecdcf92 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:41.644126 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b587b-5fca-4aad-9b4f-7e3baecdcf92 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:41.644405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b587b-5fca-4aad-9b4f-7e3baecdcf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:41.664603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-413b587b-5fca-4aad-9b4f-7e3baecdcf92 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:41.665856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-413b587b-5fca-4aad-9b4f-7e3baecdcf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:41.734443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 931/1843] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:41.743502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851877a-3433-4f39-9e90-8ef21c337960 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:41.743938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851877a-3433-4f39-9e90-8ef21c337960 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:41.744923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851877a-3433-4f39-9e90-8ef21c337960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:41.772202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-9851877a-3433-4f39-9e90-8ef21c337960 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:41.773451 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-9851877a-3433-4f39-9e90-8ef21c337960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:41.782701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 913/1844] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:41.786940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b587b-5fca-4aad-9b4f-7e3baecdcf92 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:41.805934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b587b-5fca-4aad-9b4f-7e3baecdcf92 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:41.812389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b3d76-43ea-4c3a-a04c-cb34f9c7865a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:41.813780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b3d76-43ea-4c3a-a04c-cb34f9c7865a admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:41.814184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b3d76-43ea-4c3a-a04c-cb34f9c7865a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:41.814447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b3d76-43ea-4c3a-a04c-cb34f9c7865a admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:41.815185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b3d76-43ea-4c3a-a04c-cb34f9c7865a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:56:41.815501 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b3d76-43ea-4c3a-a04c-cb34f9c7865a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:41.817360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b3d76-43ea-4c3a-a04c-cb34f9c7865a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:41.830945 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 932/1845] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:56:41.835481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851877a-3433-4f39-9e90-8ef21c337960 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:41.844699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851877a-3433-4f39-9e90-8ef21c337960 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:41.850130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de04c6-491c-46ac-b0de-9662cef69040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:41.850454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de04c6-491c-46ac-b0de-9662cef69040 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:41.850732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de04c6-491c-46ac-b0de-9662cef69040 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:41.851034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de04c6-491c-46ac-b0de-9662cef69040 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:41.851686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de04c6-491c-46ac-b0de-9662cef69040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:41.853640 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de04c6-491c-46ac-b0de-9662cef69040 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:41.979013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 914/1846] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:41 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:41.981919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b3d76-43ea-4c3a-a04c-cb34f9c7865a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:42.009454 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b3d76-43ea-4c3a-a04c-cb34f9c7865a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:42.094546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e9d4b-1264-4c3a-b376-36eac28b36d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:42.095237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e9d4b-1264-4c3a-b376-36eac28b36d5 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:42.095554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e9d4b-1264-4c3a-b376-36eac28b36d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:42.095833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e9d4b-1264-4c3a-b376-36eac28b36d5 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:42.096696 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e9d4b-1264-4c3a-b376-36eac28b36d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:42.098681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e9d4b-1264-4c3a-b376-36eac28b36d5 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:42.207206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 933/1847] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:41 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:42.211587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de04c6-491c-46ac-b0de-9662cef69040 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:42.222693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de04c6-491c-46ac-b0de-9662cef69040 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:42.227388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d1181-9ebf-434f-9966-2c40404ccffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:42.228134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d1181-9ebf-434f-9966-2c40404ccffa admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:42.228495 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d1181-9ebf-434f-9966-2c40404ccffa admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:42.228842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d1181-9ebf-434f-9966-2c40404ccffa admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:42.229802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d1181-9ebf-434f-9966-2c40404ccffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:42.231765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d1181-9ebf-434f-9966-2c40404ccffa admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:42.242781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 915/1848] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:56:42.246813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e9d4b-1264-4c3a-b376-36eac28b36d5 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:42.260598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e9d4b-1264-4c3a-b376-36eac28b36d5 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:42.267379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f896e380-e2ab-4bc7-b94c-8967b8a0cd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:42.267773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e380-e2ab-4bc7-b94c-8967b8a0cd2e admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:42.268227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e380-e2ab-4bc7-b94c-8967b8a0cd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:42.268501 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f896e380-e2ab-4bc7-b94c-8967b8a0cd2e admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:42.269222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896e380-e2ab-4bc7-b94c-8967b8a0cd2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:42.271004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f896e380-e2ab-4bc7-b94c-8967b8a0cd2e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:42.288298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 934/1849] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:42.289300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d1181-9ebf-434f-9966-2c40404ccffa admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:42.300374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d1181-9ebf-434f-9966-2c40404ccffa admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:42.308602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce913541-5a82-4391-b762-bebe4a2c138e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:42.320444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce913541-5a82-4391-b762-bebe4a2c138e admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:42.324611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce913541-5a82-4391-b762-bebe4a2c138e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:42.325502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce913541-5a82-4391-b762-bebe4a2c138e admin admin] PATH_INFO: `/v3/projects/efd4e5cb59004ef1bd185ce2daa421dd/users/cedb76af3e014abf8d55d834a3e08a46/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:42.353547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce913541-5a82-4391-b762-bebe4a2c138e admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd4e5cb59004ef1bd185ce2daa421dd, user_id=cedb76af3e014abf8d55d834a3e08a46, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:42.356487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce913541-5a82-4391-b762-bebe4a2c138e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:42.462721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 916/1850] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:42 2022] PUT /identity/v3/projects/efd4e5cb59004ef1bd185ce2daa421dd/users/cedb76af3e014abf8d55d834a3e08a46/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:42.463966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4052d7-d15c-4abd-844b-04ca47ad6d72 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:42.464291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4052d7-d15c-4abd-844b-04ca47ad6d72 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:42.464575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4052d7-d15c-4abd-844b-04ca47ad6d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:42.492554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3d4052d7-d15c-4abd-844b-04ca47ad6d72 None None] MFA Rules not processed for user `cedb76af3e014abf8d55d834a3e08a46`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:42.493830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3d4052d7-d15c-4abd-844b-04ca47ad6d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:42.530032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:42.572260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce913541-5a82-4391-b762-bebe4a2c138e admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:42.577186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ce913541-5a82-4391-b762-bebe4a2c138e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:42.610839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 935/1851] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:42.615010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4052d7-d15c-4abd-844b-04ca47ad6d72 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:42.675879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce913541-5a82-4391-b762-bebe4a2c138e admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:42.683438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c22e62-c6c6-4a9f-b36b-709e21a0ec8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:42.685096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c22e62-c6c6-4a9f-b36b-709e21a0ec8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:42.686562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c22e62-c6c6-4a9f-b36b-709e21a0ec8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:42.686562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c22e62-c6c6-4a9f-b36b-709e21a0ec8f admin admin] PATH_INFO: `/v3/users/55255f04f5d0401b96b3838f388529d3` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:42.687087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c22e62-c6c6-4a9f-b36b-709e21a0ec8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=55255f04f5d0401b96b3838f388529d3)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:42.689750 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c22e62-c6c6-4a9f-b36b-709e21a0ec8f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:42.695977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4052d7-d15c-4abd-844b-04ca47ad6d72 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:42.704119 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17733a4-d246-40bf-b5a7-dc07bf342cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:42.704796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17733a4-d246-40bf-b5a7-dc07bf342cae admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:42.705085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17733a4-d246-40bf-b5a7-dc07bf342cae admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:42.705420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17733a4-d246-40bf-b5a7-dc07bf342cae admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:42.706406 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17733a4-d246-40bf-b5a7-dc07bf342cae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:42.707756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:42.708045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17733a4-d246-40bf-b5a7-dc07bf342cae admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:42.855267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 936/1852] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:42 2022] POST /identity/v3/projects => generated 377 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:56:42.856721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17733a4-d246-40bf-b5a7-dc07bf342cae admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:42.929276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17733a4-d246-40bf-b5a7-dc07bf342cae admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:42.936859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f8a3c-0b9b-48a6-a71f-09a4a1694fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:42.939977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f8a3c-0b9b-48a6-a71f-09a4a1694fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:42.939977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f8a3c-0b9b-48a6-a71f-09a4a1694fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:42.939977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f8a3c-0b9b-48a6-a71f-09a4a1694fa6 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:42.939977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f8a3c-0b9b-48a6-a71f-09a4a1694fa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:42.940969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f8a3c-0b9b-48a6-a71f-09a4a1694fa6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:42.959647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-12c22e62-c6c6-4a9f-b36b-709e21a0ec8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55255f04f5d0401b96b3838f388529d3'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:42.967117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-12c22e62-c6c6-4a9f-b36b-709e21a0ec8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55255f04f5d0401b96b3838f388529d3'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:42.968038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-12c22e62-c6c6-4a9f-b36b-709e21a0ec8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55255f04f5d0401b96b3838f388529d3'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:42.980139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-12c22e62-c6c6-4a9f-b36b-709e21a0ec8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55255f04f5d0401b96b3838f388529d3'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:42.981087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-12c22e62-c6c6-4a9f-b36b-709e21a0ec8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55255f04f5d0401b96b3838f388529d3'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:43.030116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 937/1853] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:42 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:56:43.034645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f8a3c-0b9b-48a6-a71f-09a4a1694fa6 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:43.036355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-871f8a3c-0b9b-48a6-a71f-09a4a1694fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:43.049625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f8a3c-0b9b-48a6-a71f-09a4a1694fa6 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:43.052006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 917/1854] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:42 2022] DELETE /identity/v3/users/55255f04f5d0401b96b3838f388529d3 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:43.054488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ac572-291a-4b43-9089-3b545ddc238a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:43.055195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ac572-291a-4b43-9089-3b545ddc238a admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:43.055554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ac572-291a-4b43-9089-3b545ddc238a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:43.055838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ac572-291a-4b43-9089-3b545ddc238a admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:43.056676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ac572-291a-4b43-9089-3b545ddc238a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:43.058479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df27c8d-d18d-46fe-be70-4559b7370965 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:43.058632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ac572-291a-4b43-9089-3b545ddc238a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:43.058872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df27c8d-d18d-46fe-be70-4559b7370965 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:43.059165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df27c8d-d18d-46fe-be70-4559b7370965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:43.073271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 938/1855] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:43.081041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ac572-291a-4b43-9089-3b545ddc238a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:43.085419 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1df27c8d-d18d-46fe-be70-4559b7370965 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:43.085419 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1df27c8d-d18d-46fe-be70-4559b7370965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:43.096047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ac572-291a-4b43-9089-3b545ddc238a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:43.137779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:43.153071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 918/1856] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:43.168191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df27c8d-d18d-46fe-be70-4559b7370965 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:43.170566 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1df27c8d-d18d-46fe-be70-4559b7370965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:43.177829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23e2a1-0658-4f35-88c3-d30502b4f646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:43.178511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23e2a1-0658-4f35-88c3-d30502b4f646 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:43.178794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23e2a1-0658-4f35-88c3-d30502b4f646 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:43.179069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23e2a1-0658-4f35-88c3-d30502b4f646 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:43.180158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23e2a1-0658-4f35-88c3-d30502b4f646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:43.181841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23e2a1-0658-4f35-88c3-d30502b4f646 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:43.195912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 939/1857] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:56:43.197029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23e2a1-0658-4f35-88c3-d30502b4f646 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:43.207088 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23e2a1-0658-4f35-88c3-d30502b4f646 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:43.212352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4015375-a61b-47b9-91bc-7caa60f78683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:43.213214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4015375-a61b-47b9-91bc-7caa60f78683 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:43.213530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4015375-a61b-47b9-91bc-7caa60f78683 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:43.213804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4015375-a61b-47b9-91bc-7caa60f78683 admin admin] PATH_INFO: `/v3/projects/cb8ccba52e7244fbabc7220149426f19/users/1d93c5ef81eb49e8846f38d6b52df923/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:43.229149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4015375-a61b-47b9-91bc-7caa60f78683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8ccba52e7244fbabc7220149426f19, user_id=1d93c5ef81eb49e8846f38d6b52df923, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:43.232290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4015375-a61b-47b9-91bc-7caa60f78683 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:43.232826 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df27c8d-d18d-46fe-be70-4559b7370965 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:43.236775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebeb26-5cb9-4f72-a088-4820692ca879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:43.237513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebeb26-5cb9-4f72-a088-4820692ca879 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:43.237788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebeb26-5cb9-4f72-a088-4820692ca879 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:43.238064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebeb26-5cb9-4f72-a088-4820692ca879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:43.247323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebeb26-5cb9-4f72-a088-4820692ca879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:43.248788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebeb26-5cb9-4f72-a088-4820692ca879 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:43.258667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 919/1858] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:56:43.331606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 940/1859] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:43 2022] PUT /identity/v3/projects/cb8ccba52e7244fbabc7220149426f19/users/1d93c5ef81eb49e8846f38d6b52df923/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:43.350334 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb2f25-1db7-41e7-b774-b2c87a6788cd None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:43.350855 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb2f25-1db7-41e7-b774-b2c87a6788cd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:43.350931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb2f25-1db7-41e7-b774-b2c87a6788cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:43.399389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-5fbb2f25-1db7-41e7-b774-b2c87a6788cd None None] MFA Rules not processed for user `1d93c5ef81eb49e8846f38d6b52df923`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:43.400675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5fbb2f25-1db7-41e7-b774-b2c87a6788cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:43.434851 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:43.530678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 920/1860] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:43.535331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4015375-a61b-47b9-91bc-7caa60f78683 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:43.631858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4015375-a61b-47b9-91bc-7caa60f78683 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:43.650708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dd00c-a5c2-4ebe-b2dc-9e729dc66885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:43.667430 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dd00c-a5c2-4ebe-b2dc-9e729dc66885 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:43.667899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dd00c-a5c2-4ebe-b2dc-9e729dc66885 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:43.668178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dd00c-a5c2-4ebe-b2dc-9e729dc66885 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:43.668908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dd00c-a5c2-4ebe-b2dc-9e729dc66885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:43.671816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dd00c-a5c2-4ebe-b2dc-9e729dc66885 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:43.789128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb2f25-1db7-41e7-b774-b2c87a6788cd None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:43.790739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5fbb2f25-1db7-41e7-b774-b2c87a6788cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:43.797378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 941/1861] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:43 2022] POST /identity/v3/projects => generated 379 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:43.801225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dd00c-a5c2-4ebe-b2dc-9e729dc66885 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:43.812949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb2f25-1db7-41e7-b774-b2c87a6788cd None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:43.871081 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dd00c-a5c2-4ebe-b2dc-9e729dc66885 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:43.876719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c13c0-311d-4c23-8b43-24bf80fd1029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:43.877758 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c13c0-311d-4c23-8b43-24bf80fd1029 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:43.878048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c13c0-311d-4c23-8b43-24bf80fd1029 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:43.878326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c13c0-311d-4c23-8b43-24bf80fd1029 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:43.879015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c13c0-311d-4c23-8b43-24bf80fd1029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:43.881915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c13c0-311d-4c23-8b43-24bf80fd1029 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:43.887449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:43.889068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:43.889524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:43.889806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] PATH_INFO: `/v3/projects/d475b60042f24164a36ae00d6b137760` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:43.891605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d475b60042f24164a36ae00d6b137760)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:43.893738 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:43.963881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 942/1862] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:43 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:56:43.963881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abc6c-d071-4ab2-a727-0712e6dd2cd8 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:43.963881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abc6c-d071-4ab2-a727-0712e6dd2cd8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:43.963881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977abc6c-d071-4ab2-a727-0712e6dd2cd8 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:43.965333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 943/1863] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 18:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:56:43.966139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977abc6c-d071-4ab2-a727-0712e6dd2cd8 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:43.991779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977abc6c-d071-4ab2-a727-0712e6dd2cd8 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:44.000981 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685e4d3-630e-46aa-94cf-9b9c462568e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:44.001985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685e4d3-630e-46aa-94cf-9b9c462568e4 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:44.002275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685e4d3-630e-46aa-94cf-9b9c462568e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:44.002604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685e4d3-630e-46aa-94cf-9b9c462568e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:44.004062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685e4d3-630e-46aa-94cf-9b9c462568e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:44.008170 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685e4d3-630e-46aa-94cf-9b9c462568e4 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:44.018896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 944/1864] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:44.020268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b94eac-1703-4a23-bd58-b8486849a96f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:44.020988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b94eac-1703-4a23-bd58-b8486849a96f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:44.021316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b94eac-1703-4a23-bd58-b8486849a96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:44.035880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:44.041579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-e9b94eac-1703-4a23-bd58-b8486849a96f None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:44.042864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e9b94eac-1703-4a23-bd58-b8486849a96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:44.077034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd475b60042f24164a36ae00d6b137760', 'request_id': 'req-63a75dd6-7ae2-41c9-9945-1acfc860019e'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:44.141550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 945/1865] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 18:56:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:44.141550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b94eac-1703-4a23-bd58-b8486849a96f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:44.149221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b94eac-1703-4a23-bd58-b8486849a96f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:44.159395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd475b60042f24164a36ae00d6b137760', 'request_id': 'req-63a75dd6-7ae2-41c9-9945-1acfc860019e'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:44.168604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd475b60042f24164a36ae00d6b137760', 'request_id': 'req-63a75dd6-7ae2-41c9-9945-1acfc860019e'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:44.182843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] The token cache is being invalidate because project d475b60042f24164a36ae00d6b137760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:44.183440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:44.183807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:44.189508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 921/1866] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:43 2022] DELETE /identity/v3/projects/d475b60042f24164a36ae00d6b137760 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:56:44.190602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:44.192320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:44.253840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c7aa2-21ce-43b5-a0cd-7251d6bfb2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:44.254983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c7aa2-21ce-43b5-a0cd-7251d6bfb2f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:44.255369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c7aa2-21ce-43b5-a0cd-7251d6bfb2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:44.260523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c7aa2-21ce-43b5-a0cd-7251d6bfb2f5 admin admin] PATH_INFO: `/v3/projects/0058a08d1ee14e3a9e9baa0d3aa38126/users/86e9277f3cb348d3ba11dcfd7bbb4ce7/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:44.274063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c7aa2-21ce-43b5-a0cd-7251d6bfb2f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0058a08d1ee14e3a9e9baa0d3aa38126, user_id=86e9277f3cb348d3ba11dcfd7bbb4ce7, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:44.276754 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c7aa2-21ce-43b5-a0cd-7251d6bfb2f5 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:44.287092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a75dd6-7ae2-41c9-9945-1acfc860019e admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:44.291898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:44.292764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:44.293103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:44.293428 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] PATH_INFO: `/v3/users/a35619a6539a4b6e92b2f820af45dc5c` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:44.294682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a35619a6539a4b6e92b2f820af45dc5c)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:44.296655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:44.378955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 946/1867] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:44 2022] PUT /identity/v3/projects/0058a08d1ee14e3a9e9baa0d3aa38126/users/86e9277f3cb348d3ba11dcfd7bbb4ce7/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:44.380300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c7aa2-21ce-43b5-a0cd-7251d6bfb2f5 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:44.384696 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-527c7aa2-21ce-43b5-a0cd-7251d6bfb2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:44.469129 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c7aa2-21ce-43b5-a0cd-7251d6bfb2f5 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:44.476090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab548b6c-c205-41a7-9402-0ed91585e1fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:44.476812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab548b6c-c205-41a7-9402-0ed91585e1fd service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:44.477100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab548b6c-c205-41a7-9402-0ed91585e1fd service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:44.477412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab548b6c-c205-41a7-9402-0ed91585e1fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:44.479562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ab548b6c-c205-41a7-9402-0ed91585e1fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:44.486546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a35619a6539a4b6e92b2f820af45dc5c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:44.493682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a35619a6539a4b6e92b2f820af45dc5c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:44.494785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab548b6c-c205-41a7-9402-0ed91585e1fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:44.497119 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab548b6c-c205-41a7-9402-0ed91585e1fd service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:44.503918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a35619a6539a4b6e92b2f820af45dc5c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:44.511437 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 947/1868] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:44.512038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab548b6c-c205-41a7-9402-0ed91585e1fd service nova] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:44.514192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ab548b6c-c205-41a7-9402-0ed91585e1fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:44.529905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a35619a6539a4b6e92b2f820af45dc5c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:44.531848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a35619a6539a4b6e92b2f820af45dc5c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:44.584571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:44.602955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab548b6c-c205-41a7-9402-0ed91585e1fd service nova] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:44.609639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 922/1869] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:44 2022] DELETE /identity/v3/users/a35619a6539a4b6e92b2f820af45dc5c => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:44.631604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b024a4f-99ab-4e09-a91e-f20deafa4570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:44.634450 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b024a4f-99ab-4e09-a91e-f20deafa4570 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:44.634908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b024a4f-99ab-4e09-a91e-f20deafa4570 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:44.635205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b024a4f-99ab-4e09-a91e-f20deafa4570 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:44.636238 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b024a4f-99ab-4e09-a91e-f20deafa4570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:44.639498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b024a4f-99ab-4e09-a91e-f20deafa4570 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:44.666838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 948/1870] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:44.668668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:44.670718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:44.796551 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8209647-02cc-4c68-8155-6d71c9bede02 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:44.801255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59c5c3-ca7b-443b-a49c-e5550860ec1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:44.802224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59c5c3-ca7b-443b-a49c-e5550860ec1f admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:44.802506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59c5c3-ca7b-443b-a49c-e5550860ec1f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:44.802872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59c5c3-ca7b-443b-a49c-e5550860ec1f admin admin] PATH_INFO: `/v3/projects/0058a08d1ee14e3a9e9baa0d3aa38126/users/86e9277f3cb348d3ba11dcfd7bbb4ce7/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:44.804856 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59c5c3-ca7b-443b-a49c-e5550860ec1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0058a08d1ee14e3a9e9baa0d3aa38126, user_id=86e9277f3cb348d3ba11dcfd7bbb4ce7, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:44.807799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59c5c3-ca7b-443b-a49c-e5550860ec1f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:44.827512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 923/1871] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:44 2022] PUT /identity/v3/projects/0058a08d1ee14e3a9e9baa0d3aa38126/users/86e9277f3cb348d3ba11dcfd7bbb4ce7/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:44.837331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78847d-3f00-474a-b276-732d9efefbc1 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:44.837631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78847d-3f00-474a-b276-732d9efefbc1 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:44.837919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78847d-3f00-474a-b276-732d9efefbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:44.845923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:44.874188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-7b78847d-3f00-474a-b276-732d9efefbc1 None None] MFA Rules not processed for user `86e9277f3cb348d3ba11dcfd7bbb4ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:44.875424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7b78847d-3f00-474a-b276-732d9efefbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:44.960856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 949/1872] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:44.975036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59c5c3-ca7b-443b-a49c-e5550860ec1f admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:44.976786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-da59c5c3-ca7b-443b-a49c-e5550860ec1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:45.064521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59c5c3-ca7b-443b-a49c-e5550860ec1f admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:45.068463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830961b6-b334-4143-a557-e781f4621df1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:45.069123 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830961b6-b334-4143-a557-e781f4621df1 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:45.069443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830961b6-b334-4143-a557-e781f4621df1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:45.069723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830961b6-b334-4143-a557-e781f4621df1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:45.081014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830961b6-b334-4143-a557-e781f4621df1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:45.082736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830961b6-b334-4143-a557-e781f4621df1 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:45.093960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 924/1873] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:45.200374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78847d-3f00-474a-b276-732d9efefbc1 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:45.206563 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7b78847d-3f00-474a-b276-732d9efefbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:45.228304 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830961b6-b334-4143-a557-e781f4621df1 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:45.230065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-830961b6-b334-4143-a557-e781f4621df1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:45.247173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830961b6-b334-4143-a557-e781f4621df1 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:45.253437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9b347-ab52-4b93-a4b6-bca80ef6f20f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:45.254108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9b347-ab52-4b93-a4b6-bca80ef6f20f service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:45.254394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9b347-ab52-4b93-a4b6-bca80ef6f20f service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:45.254686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9b347-ab52-4b93-a4b6-bca80ef6f20f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:45.256983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-03e9b347-ab52-4b93-a4b6-bca80ef6f20f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:45.323735 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78847d-3f00-474a-b276-732d9efefbc1 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:45.330078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:45.330940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:45.331228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:45.331509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] PATH_INFO: `/v3/projects/b6cd7ce980fe48fbafd8aea4b8bf9ce8` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:45.333011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6cd7ce980fe48fbafd8aea4b8bf9ce8)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:45.335339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:45.345681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9b347-ab52-4b93-a4b6-bca80ef6f20f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:45.347202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9b347-ab52-4b93-a4b6-bca80ef6f20f service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:45.360638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 925/1874] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:56:45.487422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:45.506707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6cd7ce980fe48fbafd8aea4b8bf9ce8', 'request_id': 'req-46f1dd38-8f2f-4c80-9051-da3bbecf037e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:45.533096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:45.626757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6cd7ce980fe48fbafd8aea4b8bf9ce8', 'request_id': 'req-46f1dd38-8f2f-4c80-9051-da3bbecf037e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:45.635967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6cd7ce980fe48fbafd8aea4b8bf9ce8', 'request_id': 'req-46f1dd38-8f2f-4c80-9051-da3bbecf037e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:45.651963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] The token cache is being invalidate because project b6cd7ce980fe48fbafd8aea4b8bf9ce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:45.652353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:45.653114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-46f1dd38-8f2f-4c80-9051-da3bbecf037e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:45.656353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 950/1875] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:45 2022] DELETE /identity/v3/projects/b6cd7ce980fe48fbafd8aea4b8bf9ce8 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:45.661379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e5f73-da9a-4b12-93a5-9428658f8720 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:45.661860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e5f73-da9a-4b12-93a5-9428658f8720 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:45.662342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e5f73-da9a-4b12-93a5-9428658f8720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:45.689756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-bb3e5f73-da9a-4b12-93a5-9428658f8720 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:45.691021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-bb3e5f73-da9a-4b12-93a5-9428658f8720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:45.766794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 926/1876] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:45.774658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a2094-b559-4a09-8f6f-fa087bfa54d5 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:45.775000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a2094-b559-4a09-8f6f-fa087bfa54d5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:45.775282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a2094-b559-4a09-8f6f-fa087bfa54d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:45.796489 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-957a2094-b559-4a09-8f6f-fa087bfa54d5 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:45.797748 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-957a2094-b559-4a09-8f6f-fa087bfa54d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:45.811190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 951/1877] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:45.812700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e5f73-da9a-4b12-93a5-9428658f8720 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:45.828762 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e5f73-da9a-4b12-93a5-9428658f8720 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:45.834184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377baccb-65f8-4375-9b2e-4c15f1a36879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:45.834853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377baccb-65f8-4375-9b2e-4c15f1a36879 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:45.835139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377baccb-65f8-4375-9b2e-4c15f1a36879 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:45.835612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377baccb-65f8-4375-9b2e-4c15f1a36879 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:45.836334 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377baccb-65f8-4375-9b2e-4c15f1a36879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:56:45.836642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377baccb-65f8-4375-9b2e-4c15f1a36879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:45.838284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377baccb-65f8-4375-9b2e-4c15f1a36879 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:45.867451 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 927/1878] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:56:45.873149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a2094-b559-4a09-8f6f-fa087bfa54d5 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:45.882483 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a2094-b559-4a09-8f6f-fa087bfa54d5 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:45.890822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2523973-534a-454e-b589-e12b71cf8242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:45.891511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523973-534a-454e-b589-e12b71cf8242 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:45.891827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523973-534a-454e-b589-e12b71cf8242 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:45.892877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523973-534a-454e-b589-e12b71cf8242 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:45.892987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2523973-534a-454e-b589-e12b71cf8242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:45.894877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2523973-534a-454e-b589-e12b71cf8242 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:46.032544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 952/1879] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:45 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:46.036987 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377baccb-65f8-4375-9b2e-4c15f1a36879 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:46.051366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377baccb-65f8-4375-9b2e-4c15f1a36879 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:46.133812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43db459-4beb-495a-947c-71e42b1b75e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:46.134834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db459-4beb-495a-947c-71e42b1b75e9 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:46.135260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db459-4beb-495a-947c-71e42b1b75e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:46.135550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43db459-4beb-495a-947c-71e42b1b75e9 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:46.136345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43db459-4beb-495a-947c-71e42b1b75e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:46.138294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43db459-4beb-495a-947c-71e42b1b75e9 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:46.261974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 928/1880] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:46 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:46.268055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2523973-534a-454e-b589-e12b71cf8242 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:46.278361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2523973-534a-454e-b589-e12b71cf8242 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:46.283254 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591eacb-d864-4176-9aec-0791e6595861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:46.284094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591eacb-d864-4176-9aec-0791e6595861 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:46.284389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591eacb-d864-4176-9aec-0791e6595861 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:46.284670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591eacb-d864-4176-9aec-0791e6595861 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:46.285431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591eacb-d864-4176-9aec-0791e6595861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:46.287487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591eacb-d864-4176-9aec-0791e6595861 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:46.302087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 953/1881] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:46.306121 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43db459-4beb-495a-947c-71e42b1b75e9 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:46.316444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43db459-4beb-495a-947c-71e42b1b75e9 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:46.320969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755bb659-57a1-4173-889f-67975053da10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:46.321658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bb659-57a1-4173-889f-67975053da10 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:46.321999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bb659-57a1-4173-889f-67975053da10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:46.322207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bb659-57a1-4173-889f-67975053da10 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:46.322900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bb659-57a1-4173-889f-67975053da10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:46.324513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755bb659-57a1-4173-889f-67975053da10 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:46.334728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 929/1882] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:46.338678 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591eacb-d864-4176-9aec-0791e6595861 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:46.366417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591eacb-d864-4176-9aec-0791e6595861 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:46.371662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488453b5-d7c9-403e-a015-b8362ea27c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:46.372621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488453b5-d7c9-403e-a015-b8362ea27c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:46.372910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488453b5-d7c9-403e-a015-b8362ea27c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:46.373191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488453b5-d7c9-403e-a015-b8362ea27c20 admin admin] PATH_INFO: `/v3/projects/cd3ab83fd25c4770be52696711216c6c/users/90462f9da54c45c380eb9044bd46b5fe/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:46.394216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488453b5-d7c9-403e-a015-b8362ea27c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3ab83fd25c4770be52696711216c6c, user_id=90462f9da54c45c380eb9044bd46b5fe, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:46.399915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488453b5-d7c9-403e-a015-b8362ea27c20 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:46.541504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 954/1883] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:46 2022] PUT /identity/v3/projects/cd3ab83fd25c4770be52696711216c6c/users/90462f9da54c45c380eb9044bd46b5fe/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:46.552912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0103a-d15e-49b0-a303-c4eeb4b9772a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:46.553445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0103a-d15e-49b0-a303-c4eeb4b9772a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:46.553547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0103a-d15e-49b0-a303-c4eeb4b9772a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:46.604480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-e3b0103a-d15e-49b0-a303-c4eeb4b9772a None None] MFA Rules not processed for user `90462f9da54c45c380eb9044bd46b5fe`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:46.606103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e3b0103a-d15e-49b0-a303-c4eeb4b9772a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:46.639968 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:46.711479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 930/1884] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:46.834362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488453b5-d7c9-403e-a015-b8362ea27c20 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:46.836356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-488453b5-d7c9-403e-a015-b8362ea27c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:46.905599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488453b5-d7c9-403e-a015-b8362ea27c20 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:46.910344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960d345-948f-46ad-9f0e-7f2c1ff2fd89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:46.911144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960d345-948f-46ad-9f0e-7f2c1ff2fd89 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:46.911438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960d345-948f-46ad-9f0e-7f2c1ff2fd89 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:46.915180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960d345-948f-46ad-9f0e-7f2c1ff2fd89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:46.928301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960d345-948f-46ad-9f0e-7f2c1ff2fd89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:46.929986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960d345-948f-46ad-9f0e-7f2c1ff2fd89 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:46.939955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 955/1885] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:50.198817 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0103a-d15e-49b0-a303-c4eeb4b9772a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:50.200779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e3b0103a-d15e-49b0-a303-c4eeb4b9772a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:50.277688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0103a-d15e-49b0-a303-c4eeb4b9772a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:50.283044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:50.283909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:50.284189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:50.284458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] PATH_INFO: `/v3/users/cedb76af3e014abf8d55d834a3e08a46` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:50.285667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cedb76af3e014abf8d55d834a3e08a46)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:50.287656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:50.443942 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cedb76af3e014abf8d55d834a3e08a46'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:50.452400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cedb76af3e014abf8d55d834a3e08a46'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:50.453170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cedb76af3e014abf8d55d834a3e08a46'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:50.468217 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cedb76af3e014abf8d55d834a3e08a46'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:50.468933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cedb76af3e014abf8d55d834a3e08a46'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:50.575374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 931/1886] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:50 2022] DELETE /identity/v3/users/cedb76af3e014abf8d55d834a3e08a46 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:50.580161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd0377-f37d-41bb-aec9-fcc01f66d9d2 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:50.580648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd0377-f37d-41bb-aec9-fcc01f66d9d2 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:50.580725 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd0377-f37d-41bb-aec9-fcc01f66d9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:50.614827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9dcd0377-f37d-41bb-aec9-fcc01f66d9d2 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:50.616096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9dcd0377-f37d-41bb-aec9-fcc01f66d9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:50.681921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:50.695029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 956/1887] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:50.706475 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:50.707981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:50.737269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:50.771107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f5ab9-91b9-41a5-9dc1-207b7942fbe0 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:50.775569 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d1fd6-a51e-4fa8-9922-4baad2f0d57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:50.776308 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d1fd6-a51e-4fa8-9922-4baad2f0d57b service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:50.776624 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d1fd6-a51e-4fa8-9922-4baad2f0d57b service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:50.780672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d1fd6-a51e-4fa8-9922-4baad2f0d57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:50.791471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d1fd6-a51e-4fa8-9922-4baad2f0d57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:50.796133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d1fd6-a51e-4fa8-9922-4baad2f0d57b service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:50.808286 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 932/1888] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:51.951744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd0377-f37d-41bb-aec9-fcc01f66d9d2 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:51.953504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9dcd0377-f37d-41bb-aec9-fcc01f66d9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:51.966912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd0377-f37d-41bb-aec9-fcc01f66d9d2 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:51.972490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:51.973591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:51.973875 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:51.974142 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] PATH_INFO: `/v3/projects/efd4e5cb59004ef1bd185ce2daa421dd` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:51.975595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] RBAC: Authorizing `identity:delete_project(project_id=efd4e5cb59004ef1bd185ce2daa421dd)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:51.979444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:52.119624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efd4e5cb59004ef1bd185ce2daa421dd', 'request_id': 'req-f6a986af-c5ff-4bde-87d8-00488878701a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:52.164132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efd4e5cb59004ef1bd185ce2daa421dd', 'request_id': 'req-f6a986af-c5ff-4bde-87d8-00488878701a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:52.172499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efd4e5cb59004ef1bd185ce2daa421dd', 'request_id': 'req-f6a986af-c5ff-4bde-87d8-00488878701a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:52.187094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] The token cache is being invalidate because project efd4e5cb59004ef1bd185ce2daa421dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:52.187390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:52.188502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:52.190154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 957/1889] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:51 2022] DELETE /identity/v3/projects/efd4e5cb59004ef1bd185ce2daa421dd => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:52.194443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d1fd6-a51e-4fa8-9922-4baad2f0d57b service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:52.205769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d1fd6-a51e-4fa8-9922-4baad2f0d57b service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:52.310050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92196489-5a29-4156-9980-090026d69715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:52.313807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92196489-5a29-4156-9980-090026d69715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:52.314599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92196489-5a29-4156-9980-090026d69715 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:52.315357 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92196489-5a29-4156-9980-090026d69715 admin admin] PATH_INFO: `/v3/users/1d93c5ef81eb49e8846f38d6b52df923` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:52.317062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92196489-5a29-4156-9980-090026d69715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d93c5ef81eb49e8846f38d6b52df923)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:52.319970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92196489-5a29-4156-9980-090026d69715 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:52.732393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-92196489-5a29-4156-9980-090026d69715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d93c5ef81eb49e8846f38d6b52df923'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:52.732393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-92196489-5a29-4156-9980-090026d69715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d93c5ef81eb49e8846f38d6b52df923'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:52.732393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-92196489-5a29-4156-9980-090026d69715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d93c5ef81eb49e8846f38d6b52df923'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:52.732393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-92196489-5a29-4156-9980-090026d69715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d93c5ef81eb49e8846f38d6b52df923'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:52.732393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-92196489-5a29-4156-9980-090026d69715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d93c5ef81eb49e8846f38d6b52df923'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:53.080077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 933/1890] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:52 2022] DELETE /identity/v3/users/1d93c5ef81eb49e8846f38d6b52df923 => generated 0 bytes in 886 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:53.636142 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:53.636142 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:53.653015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a986af-c5ff-4bde-87d8-00488878701a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:53.736076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:53.737196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:53.737664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:53.738101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] PATH_INFO: `/v3/projects/cb8ccba52e7244fbabc7220149426f19` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:53.740083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8ccba52e7244fbabc7220149426f19)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:53.742815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:53.855959 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8ccba52e7244fbabc7220149426f19', 'request_id': 'req-8371c63a-7c1e-4b40-9581-df588714bb66'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:53.946740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8ccba52e7244fbabc7220149426f19', 'request_id': 'req-8371c63a-7c1e-4b40-9581-df588714bb66'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:53.961468 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8ccba52e7244fbabc7220149426f19', 'request_id': 'req-8371c63a-7c1e-4b40-9581-df588714bb66'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:53.988408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] The token cache is being invalidate because project cb8ccba52e7244fbabc7220149426f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:53.988801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:53.989928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:54.001789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 958/1891] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:53 2022] DELETE /identity/v3/projects/cb8ccba52e7244fbabc7220149426f19 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:54.005812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92196489-5a29-4156-9980-090026d69715 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:54.007382 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-92196489-5a29-4156-9980-090026d69715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:54.023790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92196489-5a29-4156-9980-090026d69715 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:54.085789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:54.100896 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f86c69-6e52-4770-a651-483ca61cdb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:54.101453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f86c69-6e52-4770-a651-483ca61cdb28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:54.101749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f86c69-6e52-4770-a651-483ca61cdb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:54.102082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f86c69-6e52-4770-a651-483ca61cdb28 admin admin] PATH_INFO: `/v3/users/86e9277f3cb348d3ba11dcfd7bbb4ce7` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:54.103560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f86c69-6e52-4770-a651-483ca61cdb28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86e9277f3cb348d3ba11dcfd7bbb4ce7)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:54.106049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f86c69-6e52-4770-a651-483ca61cdb28 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:54.288236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-95f86c69-6e52-4770-a651-483ca61cdb28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86e9277f3cb348d3ba11dcfd7bbb4ce7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:54.296223 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-95f86c69-6e52-4770-a651-483ca61cdb28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86e9277f3cb348d3ba11dcfd7bbb4ce7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:54.297176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-95f86c69-6e52-4770-a651-483ca61cdb28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86e9277f3cb348d3ba11dcfd7bbb4ce7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:54.313595 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-95f86c69-6e52-4770-a651-483ca61cdb28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86e9277f3cb348d3ba11dcfd7bbb4ce7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:54.314296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-95f86c69-6e52-4770-a651-483ca61cdb28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86e9277f3cb348d3ba11dcfd7bbb4ce7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:54.410689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 934/1892] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:54 2022] DELETE /identity/v3/users/86e9277f3cb348d3ba11dcfd7bbb4ce7 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:54.869843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:54.872322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:54.890008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8371c63a-7c1e-4b40-9581-df588714bb66 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:54.895285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:54.948130 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:54.949002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:54.949314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:54.949592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] PATH_INFO: `/v3/projects/0058a08d1ee14e3a9e9baa0d3aa38126` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:54.951339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0058a08d1ee14e3a9e9baa0d3aa38126)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:54.953273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:55.135843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0058a08d1ee14e3a9e9baa0d3aa38126', 'request_id': 'req-b9c61ade-9f25-4c26-a242-02a09bea2d3a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:55.204833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0058a08d1ee14e3a9e9baa0d3aa38126', 'request_id': 'req-b9c61ade-9f25-4c26-a242-02a09bea2d3a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:55.217789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0058a08d1ee14e3a9e9baa0d3aa38126', 'request_id': 'req-b9c61ade-9f25-4c26-a242-02a09bea2d3a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:55.243288 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] The token cache is being invalidate because project 0058a08d1ee14e3a9e9baa0d3aa38126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:55.243998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:55.246503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b9c61ade-9f25-4c26-a242-02a09bea2d3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:55.251938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 959/1893] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:54 2022] DELETE /identity/v3/projects/0058a08d1ee14e3a9e9baa0d3aa38126 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:55.254667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5abe2-2b8a-4501-9bbe-17dd0d6b9338 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:55.255171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5abe2-2b8a-4501-9bbe-17dd0d6b9338 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:55.255251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5abe2-2b8a-4501-9bbe-17dd0d6b9338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:55.279893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-20d5abe2-2b8a-4501-9bbe-17dd0d6b9338 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:55.281175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-20d5abe2-2b8a-4501-9bbe-17dd0d6b9338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:55.412214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 935/1894] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:55.422204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782540d-76fb-48ec-b137-1efe623ec1cd None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:55.422761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782540d-76fb-48ec-b137-1efe623ec1cd None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:55.423193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782540d-76fb-48ec-b137-1efe623ec1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:55.445714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-1782540d-76fb-48ec-b137-1efe623ec1cd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:55.448342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-1782540d-76fb-48ec-b137-1efe623ec1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:55.459056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 960/1895] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:55.475991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782540d-76fb-48ec-b137-1efe623ec1cd None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:55.476331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5abe2-2b8a-4501-9bbe-17dd0d6b9338 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:55.482872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-20d5abe2-2b8a-4501-9bbe-17dd0d6b9338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:55.498537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782540d-76fb-48ec-b137-1efe623ec1cd None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:55.502142 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5abe2-2b8a-4501-9bbe-17dd0d6b9338 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:55.504646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31767a1e-9454-4aa8-aa8f-8fe64a02c3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:55.507767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92457da2-4926-4b00-ac02-91ae1a159bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:55.509819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92457da2-4926-4b00-ac02-91ae1a159bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:55.509819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92457da2-4926-4b00-ac02-91ae1a159bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:55.509819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92457da2-4926-4b00-ac02-91ae1a159bb6 admin admin] PATH_INFO: `/v3/users/90462f9da54c45c380eb9044bd46b5fe` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:55.511070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92457da2-4926-4b00-ac02-91ae1a159bb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90462f9da54c45c380eb9044bd46b5fe)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:55.511664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31767a1e-9454-4aa8-aa8f-8fe64a02c3dd admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:55.511974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31767a1e-9454-4aa8-aa8f-8fe64a02c3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:55.512739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31767a1e-9454-4aa8-aa8f-8fe64a02c3dd admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:55.513739 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92457da2-4926-4b00-ac02-91ae1a159bb6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:55.515905 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31767a1e-9454-4aa8-aa8f-8fe64a02c3dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:56:55.516344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31767a1e-9454-4aa8-aa8f-8fe64a02c3dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:55.521798 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31767a1e-9454-4aa8-aa8f-8fe64a02c3dd admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:55.553777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 961/1896] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:56:55.560741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31767a1e-9454-4aa8-aa8f-8fe64a02c3dd admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:55.575204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31767a1e-9454-4aa8-aa8f-8fe64a02c3dd admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:55.580985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee92125-64dd-4051-9edc-65e5576be43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:55.581949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee92125-64dd-4051-9edc-65e5576be43f admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:55.582379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee92125-64dd-4051-9edc-65e5576be43f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:55.582808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee92125-64dd-4051-9edc-65e5576be43f admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:55.583749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee92125-64dd-4051-9edc-65e5576be43f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:55.585869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee92125-64dd-4051-9edc-65e5576be43f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:55.694387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 962/1897] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:55 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:55.698584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee92125-64dd-4051-9edc-65e5576be43f admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:55.743170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-92457da2-4926-4b00-ac02-91ae1a159bb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90462f9da54c45c380eb9044bd46b5fe'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:55.751476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-92457da2-4926-4b00-ac02-91ae1a159bb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90462f9da54c45c380eb9044bd46b5fe'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:55.752193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-92457da2-4926-4b00-ac02-91ae1a159bb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90462f9da54c45c380eb9044bd46b5fe'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:55.758172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:55.766626 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-92457da2-4926-4b00-ac02-91ae1a159bb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90462f9da54c45c380eb9044bd46b5fe'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:55.767314 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-92457da2-4926-4b00-ac02-91ae1a159bb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90462f9da54c45c380eb9044bd46b5fe'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:55.771036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:55.774955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee92125-64dd-4051-9edc-65e5576be43f admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:55.780885 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def3c60-8fe6-448b-b39c-18ad915cccbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:55.783990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def3c60-8fe6-448b-b39c-18ad915cccbe admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:55.783990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def3c60-8fe6-448b-b39c-18ad915cccbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:55.783990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def3c60-8fe6-448b-b39c-18ad915cccbe admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:55.783990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def3c60-8fe6-448b-b39c-18ad915cccbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:55.786565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def3c60-8fe6-448b-b39c-18ad915cccbe admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:55.858124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 936/1898] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:55 2022] DELETE /identity/v3/users/90462f9da54c45c380eb9044bd46b5fe => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:55.862632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d439811-a5e8-4b1e-b50d-74665bc00e17 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:55.863001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d439811-a5e8-4b1e-b50d-74665bc00e17 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:55.863352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d439811-a5e8-4b1e-b50d-74665bc00e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:55.881657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-8d439811-a5e8-4b1e-b50d-74665bc00e17 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:55.882135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8d439811-a5e8-4b1e-b50d-74665bc00e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:55.912611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 963/1899] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:55 2022] POST /identity/v3/users => generated 292 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:56:55.914086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def3c60-8fe6-448b-b39c-18ad915cccbe admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:55.916176 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7def3c60-8fe6-448b-b39c-18ad915cccbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:55.937861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def3c60-8fe6-448b-b39c-18ad915cccbe admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:55.967134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 937/1900] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:55.974800 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d439811-a5e8-4b1e-b50d-74665bc00e17 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:55.976411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8d439811-a5e8-4b1e-b50d-74665bc00e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:56.010151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e75b40-bf2f-4641-99a8-8a55c306b0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:56.010919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e75b40-bf2f-4641-99a8-8a55c306b0eb admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:56.011785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e75b40-bf2f-4641-99a8-8a55c306b0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:56.012145 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e75b40-bf2f-4641-99a8-8a55c306b0eb admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:56.013655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e75b40-bf2f-4641-99a8-8a55c306b0eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:56.016262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e75b40-bf2f-4641-99a8-8a55c306b0eb admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:56.033773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 964/1901] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:56.035604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e75b40-bf2f-4641-99a8-8a55c306b0eb admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:56.051360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d439811-a5e8-4b1e-b50d-74665bc00e17 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:56.056869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815989d-bdd1-4e60-a427-0c0f3893d818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:56.057668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815989d-bdd1-4e60-a427-0c0f3893d818 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:56.064138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815989d-bdd1-4e60-a427-0c0f3893d818 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:56.064138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815989d-bdd1-4e60-a427-0c0f3893d818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:56.064621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e75b40-bf2f-4641-99a8-8a55c306b0eb admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:56.067671 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41d9be-3096-43dc-ba41-707845b9b00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:56.068837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41d9be-3096-43dc-ba41-707845b9b00c admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:56.069146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41d9be-3096-43dc-ba41-707845b9b00c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:56.069577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41d9be-3096-43dc-ba41-707845b9b00c admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:56.071015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41d9be-3096-43dc-ba41-707845b9b00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:56.072631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815989d-bdd1-4e60-a427-0c0f3893d818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:56.072759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41d9be-3096-43dc-ba41-707845b9b00c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:56.074202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815989d-bdd1-4e60-a427-0c0f3893d818 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:56.083605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 965/1902] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:56:56.087100 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 938/1903] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:56:56.087923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41d9be-3096-43dc-ba41-707845b9b00c admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:56.098215 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41d9be-3096-43dc-ba41-707845b9b00c admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:56.103039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb99f9-cff8-41b2-8e3d-650f9a213ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:56.103928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb99f9-cff8-41b2-8e3d-650f9a213ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:56.104218 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb99f9-cff8-41b2-8e3d-650f9a213ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:56.104499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb99f9-cff8-41b2-8e3d-650f9a213ec3 admin admin] PATH_INFO: `/v3/projects/95de507f5dd843fc8580d1b5e6fba8a9/users/16346205c86742c1bc99c6e4dee30941/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:56.121397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb99f9-cff8-41b2-8e3d-650f9a213ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95de507f5dd843fc8580d1b5e6fba8a9, user_id=16346205c86742c1bc99c6e4dee30941, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:56.124809 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb99f9-cff8-41b2-8e3d-650f9a213ec3 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:56.256725 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 966/1904] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:56 2022] PUT /identity/v3/projects/95de507f5dd843fc8580d1b5e6fba8a9/users/16346205c86742c1bc99c6e4dee30941/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:56:56.268863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc622e-496a-4fd4-8bc8-6fba5f9a7176 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:56.269363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc622e-496a-4fd4-8bc8-6fba5f9a7176 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:56.269737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc622e-496a-4fd4-8bc8-6fba5f9a7176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:56.300780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-23dc622e-496a-4fd4-8bc8-6fba5f9a7176 None None] MFA Rules not processed for user `16346205c86742c1bc99c6e4dee30941`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:56.302490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-23dc622e-496a-4fd4-8bc8-6fba5f9a7176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:56.412710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 939/1905] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:56.425867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb99f9-cff8-41b2-8e3d-650f9a213ec3 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:56.427927 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-dffb99f9-cff8-41b2-8e3d-650f9a213ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:56.500768 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb99f9-cff8-41b2-8e3d-650f9a213ec3 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:56.504953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0a7a4-1093-4616-8c2c-3e01e08e8dfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:56.505344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0a7a4-1093-4616-8c2c-3e01e08e8dfd service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:56.505698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0a7a4-1093-4616-8c2c-3e01e08e8dfd service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:56.505957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0a7a4-1093-4616-8c2c-3e01e08e8dfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:56.515195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0a7a4-1093-4616-8c2c-3e01e08e8dfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:56.516636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c0a7a4-1093-4616-8c2c-3e01e08e8dfd service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:56.526248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 967/1906] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:56.567165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc622e-496a-4fd4-8bc8-6fba5f9a7176 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:56.584146 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:56.656813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc622e-496a-4fd4-8bc8-6fba5f9a7176 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:56.661395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86b7c5-6c45-4f16-aae3-6b3e01ab76cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:56.662271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86b7c5-6c45-4f16-aae3-6b3e01ab76cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:56.662548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86b7c5-6c45-4f16-aae3-6b3e01ab76cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:56.662984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86b7c5-6c45-4f16-aae3-6b3e01ab76cd admin admin] PATH_INFO: `/v3/users/16346205c86742c1bc99c6e4dee30941` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:56.664208 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86b7c5-6c45-4f16-aae3-6b3e01ab76cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=16346205c86742c1bc99c6e4dee30941)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:56.666166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86b7c5-6c45-4f16-aae3-6b3e01ab76cd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:56.692553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0a7a4-1093-4616-8c2c-3e01e08e8dfd service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:56.693440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e0c0a7a4-1093-4616-8c2c-3e01e08e8dfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:56.711765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0a7a4-1093-4616-8c2c-3e01e08e8dfd service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:56.721432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:56.722355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:56.722742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:56.723237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] PATH_INFO: `/v3/projects/cd3ab83fd25c4770be52696711216c6c` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:56.724771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd3ab83fd25c4770be52696711216c6c)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:56.727168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:56.885030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:56.889030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd3ab83fd25c4770be52696711216c6c', 'request_id': 'req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:56.890525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4f86b7c5-6c45-4f16-aae3-6b3e01ab76cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16346205c86742c1bc99c6e4dee30941'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:56.897735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4f86b7c5-6c45-4f16-aae3-6b3e01ab76cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16346205c86742c1bc99c6e4dee30941'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:56.898453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4f86b7c5-6c45-4f16-aae3-6b3e01ab76cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16346205c86742c1bc99c6e4dee30941'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:56.913646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4f86b7c5-6c45-4f16-aae3-6b3e01ab76cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16346205c86742c1bc99c6e4dee30941'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:56.914560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4f86b7c5-6c45-4f16-aae3-6b3e01ab76cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16346205c86742c1bc99c6e4dee30941'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:57.001804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd3ab83fd25c4770be52696711216c6c', 'request_id': 'req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:57.025284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd3ab83fd25c4770be52696711216c6c', 'request_id': 'req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:57.055315 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] The token cache is being invalidate because project cd3ab83fd25c4770be52696711216c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:57.055614 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:57.058034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-e2e76661-18e9-42a1-8abb-f9f4fd1b98fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:57.060819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 968/1907] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:56 2022] DELETE /identity/v3/projects/cd3ab83fd25c4770be52696711216c6c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:56:57.066876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f9fa2-5533-4984-ade5-699fa3e2e74c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.067398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f9fa2-5533-4984-ade5-699fa3e2e74c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.067843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f9fa2-5533-4984-ade5-699fa3e2e74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.077078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 940/1908] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:56 2022] DELETE /identity/v3/users/16346205c86742c1bc99c6e4dee30941 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:56:57.079797 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511556d-dd5d-427d-bf5d-a56158013634 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.080204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511556d-dd5d-427d-bf5d-a56158013634 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.080707 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511556d-dd5d-427d-bf5d-a56158013634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.093141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f77f9fa2-5533-4984-ade5-699fa3e2e74c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:57.096578 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f77f9fa2-5533-4984-ade5-699fa3e2e74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:57.113938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2511556d-dd5d-427d-bf5d-a56158013634 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:57.115274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2511556d-dd5d-427d-bf5d-a56158013634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:57.181087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 941/1909] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:57.194187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 969/1910] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:57.194286 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511556d-dd5d-427d-bf5d-a56158013634 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:57.197123 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2511556d-dd5d-427d-bf5d-a56158013634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:57.209923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041e7f4-8e1b-4b9a-a248-b8642b09ca8c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.211382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041e7f4-8e1b-4b9a-a248-b8642b09ca8c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.217597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041e7f4-8e1b-4b9a-a248-b8642b09ca8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.252886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d041e7f4-8e1b-4b9a-a248-b8642b09ca8c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:57.256555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d041e7f4-8e1b-4b9a-a248-b8642b09ca8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:57.259751 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511556d-dd5d-427d-bf5d-a56158013634 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:57.265138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be90e2-72ee-4ee1-8051-17995b821b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:57.265909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be90e2-72ee-4ee1-8051-17995b821b84 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.266201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be90e2-72ee-4ee1-8051-17995b821b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.266478 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be90e2-72ee-4ee1-8051-17995b821b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.278648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 970/1911] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:57.280305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041e7f4-8e1b-4b9a-a248-b8642b09ca8c None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:57.281844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be90e2-72ee-4ee1-8051-17995b821b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:57.286590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28be90e2-72ee-4ee1-8051-17995b821b84 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:57.294650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041e7f4-8e1b-4b9a-a248-b8642b09ca8c None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:57.300953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e0a27-1c95-479c-b7d0-1ea49cc264fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:57.302106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e0a27-1c95-479c-b7d0-1ea49cc264fa admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.308226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e0a27-1c95-479c-b7d0-1ea49cc264fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.308355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 942/1912] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:57.311268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e0a27-1c95-479c-b7d0-1ea49cc264fa admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.317834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e0a27-1c95-479c-b7d0-1ea49cc264fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:56:57.318159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e0a27-1c95-479c-b7d0-1ea49cc264fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:57.319895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e0a27-1c95-479c-b7d0-1ea49cc264fa admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:57.333653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 971/1913] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:56:57.337953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be90e2-72ee-4ee1-8051-17995b821b84 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:57.355075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be90e2-72ee-4ee1-8051-17995b821b84 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:57.363051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099afc5a-62ad-4e57-963d-e2e0a7960a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:57.363731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099afc5a-62ad-4e57-963d-e2e0a7960a8c admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.364180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099afc5a-62ad-4e57-963d-e2e0a7960a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.364665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099afc5a-62ad-4e57-963d-e2e0a7960a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.365536 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099afc5a-62ad-4e57-963d-e2e0a7960a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:57.367643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099afc5a-62ad-4e57-963d-e2e0a7960a8c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:57.441817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e0a27-1c95-479c-b7d0-1ea49cc264fa admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:57.443445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a09e0a27-1c95-479c-b7d0-1ea49cc264fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:57.456239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e0a27-1c95-479c-b7d0-1ea49cc264fa admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:57.460867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:57.461767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.462056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.462335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] PATH_INFO: `/v3/users/27fb9df11b87426795f9f12d5c2eb86a` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.463527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27fb9df11b87426795f9f12d5c2eb86a)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:57.465621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:57.502431 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 943/1914] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:57 2022] POST /identity/v3/projects => generated 367 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:56:57.506294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099afc5a-62ad-4e57-963d-e2e0a7960a8c admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:57.565289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099afc5a-62ad-4e57-963d-e2e0a7960a8c admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:57.569976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba67f00-268b-4e3d-81ec-67f4a2df37b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:57.570648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba67f00-268b-4e3d-81ec-67f4a2df37b4 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.570945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba67f00-268b-4e3d-81ec-67f4a2df37b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.571207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba67f00-268b-4e3d-81ec-67f4a2df37b4 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.571903 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba67f00-268b-4e3d-81ec-67f4a2df37b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:57.573937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba67f00-268b-4e3d-81ec-67f4a2df37b4 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:57.633057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 944/1915] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:57 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:56:57.637108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba67f00-268b-4e3d-81ec-67f4a2df37b4 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:57.647083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba67f00-268b-4e3d-81ec-67f4a2df37b4 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:57.651982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f11349-6e2c-460b-9793-217174e5fa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:57.652650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11349-6e2c-460b-9793-217174e5fa6e admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.652980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11349-6e2c-460b-9793-217174e5fa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.653284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11349-6e2c-460b-9793-217174e5fa6e admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.654045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f11349-6e2c-460b-9793-217174e5fa6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:57.655844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f11349-6e2c-460b-9793-217174e5fa6e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:57.666293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 945/1916] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:56:57.670686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f11349-6e2c-460b-9793-217174e5fa6e admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:57.674547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27fb9df11b87426795f9f12d5c2eb86a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:57.675367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27fb9df11b87426795f9f12d5c2eb86a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:57.680990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f11349-6e2c-460b-9793-217174e5fa6e admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:57.683584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27fb9df11b87426795f9f12d5c2eb86a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:57.688078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27fb9df11b87426795f9f12d5c2eb86a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:57.688195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd111bc-3d86-4c22-a8f2-3c73aa9530a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:57.688195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd111bc-3d86-4c22-a8f2-3c73aa9530a1 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.688195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd111bc-3d86-4c22-a8f2-3c73aa9530a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.688195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd111bc-3d86-4c22-a8f2-3c73aa9530a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.688784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd111bc-3d86-4c22-a8f2-3c73aa9530a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:57.690600 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd111bc-3d86-4c22-a8f2-3c73aa9530a1 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:57.702274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 946/1917] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:56:57.711342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27fb9df11b87426795f9f12d5c2eb86a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:57.711525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd111bc-3d86-4c22-a8f2-3c73aa9530a1 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:57.711525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1cd111bc-3d86-4c22-a8f2-3c73aa9530a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:57.721521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd111bc-3d86-4c22-a8f2-3c73aa9530a1 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:57.726101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5b6f7-7f97-4de0-9383-c0795ccaefc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:57.726977 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5b6f7-7f97-4de0-9383-c0795ccaefc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.727254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5b6f7-7f97-4de0-9383-c0795ccaefc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.727631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5b6f7-7f97-4de0-9383-c0795ccaefc4 admin admin] PATH_INFO: `/v3/projects/2c53f45e2ea14ae9b8c82aa3fa271bc1/users/53ae3beaebb3421e9cf46068f0291f16/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.743110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5b6f7-7f97-4de0-9383-c0795ccaefc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c53f45e2ea14ae9b8c82aa3fa271bc1, user_id=53ae3beaebb3421e9cf46068f0291f16, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:57.745787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5b6f7-7f97-4de0-9383-c0795ccaefc4 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:57.828105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 972/1918] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:57 2022] DELETE /identity/v3/users/27fb9df11b87426795f9f12d5c2eb86a => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:57.829383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 947/1919] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:57 2022] PUT /identity/v3/projects/2c53f45e2ea14ae9b8c82aa3fa271bc1/users/53ae3beaebb3421e9cf46068f0291f16/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:57.829595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:57.831098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:57.832757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1f84f-e356-40df-a5f5-be8d022c6780 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.833048 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1f84f-e356-40df-a5f5-be8d022c6780 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.833368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1f84f-e356-40df-a5f5-be8d022c6780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.851853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2cc1f84f-e356-40df-a5f5-be8d022c6780 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:57.859589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2cc1f84f-e356-40df-a5f5-be8d022c6780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:57.899293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c16a2e-3c8c-44bc-8b19-138823277116 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:57.904516 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:57.905339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.905622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.906031 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] PATH_INFO: `/v3/projects/95de507f5dd843fc8580d1b5e6fba8a9` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.907219 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95de507f5dd843fc8580d1b5e6fba8a9)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:57.909125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:57.914847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 948/1920] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:57.916940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163da1-28de-4c12-bcee-4c916bac0558 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:57.916940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163da1-28de-4c12-bcee-4c916bac0558 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:57.916940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2163da1-28de-4c12-bcee-4c916bac0558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:57.944847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b2163da1-28de-4c12-bcee-4c916bac0558 None None] MFA Rules not processed for user `53ae3beaebb3421e9cf46068f0291f16`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:57.946075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b2163da1-28de-4c12-bcee-4c916bac0558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:57.981925 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:58.010428 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 949/1921] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:58.011158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163da1-28de-4c12-bcee-4c916bac0558 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:58.012496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b2163da1-28de-4c12-bcee-4c916bac0558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:58.049632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95de507f5dd843fc8580d1b5e6fba8a9', 'request_id': 'req-7a617e50-de88-40aa-894b-b82766decfa7'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:58.082113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2163da1-28de-4c12-bcee-4c916bac0558 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:58.129300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95de507f5dd843fc8580d1b5e6fba8a9', 'request_id': 'req-7a617e50-de88-40aa-894b-b82766decfa7'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:58.130849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa0f08-b89d-4e8b-9c58-df7e0ccd8ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:58.131502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa0f08-b89d-4e8b-9c58-df7e0ccd8ebd service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.131790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa0f08-b89d-4e8b-9c58-df7e0ccd8ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.132065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa0f08-b89d-4e8b-9c58-df7e0ccd8ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.135817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95de507f5dd843fc8580d1b5e6fba8a9', 'request_id': 'req-7a617e50-de88-40aa-894b-b82766decfa7'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:58.147785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] The token cache is being invalidate because project 95de507f5dd843fc8580d1b5e6fba8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:58.148074 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:58.148744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:58.150617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 973/1922] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:57 2022] DELETE /identity/v3/projects/95de507f5dd843fc8580d1b5e6fba8a9 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:58.151374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:58.152683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:58.193579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa0f08-b89d-4e8b-9c58-df7e0ccd8ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:58.195130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa0f08-b89d-4e8b-9c58-df7e0ccd8ebd service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:58.207015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 950/1923] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:58.208603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3d2d8-5f14-4355-808a-90f6d4b7e3f9 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.208721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3d2d8-5f14-4355-808a-90f6d4b7e3f9 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.208928 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3d2d8-5f14-4355-808a-90f6d4b7e3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.212099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a617e50-de88-40aa-894b-b82766decfa7 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:58.217128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60647cb1-8a00-4129-b13a-e5497c3a2474 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17b7a7128d64b2ba548add02d3ec264', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:58.218082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60647cb1-8a00-4129-b13a-e5497c3a2474 service glance] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.218407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60647cb1-8a00-4129-b13a-e5497c3a2474 service glance] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.218716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60647cb1-8a00-4129-b13a-e5497c3a2474 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.222679 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-60647cb1-8a00-4129-b13a-e5497c3a2474 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:58.232171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-48f3d2d8-5f14-4355-808a-90f6d4b7e3f9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:58.233401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-48f3d2d8-5f14-4355-808a-90f6d4b7e3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:58.247274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 951/1924] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:58.255710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5b64b-5110-4bb8-bafd-34a347190566 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.256241 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5b64b-5110-4bb8-bafd-34a347190566 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.256529 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5b64b-5110-4bb8-bafd-34a347190566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.257027 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:58.278762 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-efe5b64b-5110-4bb8-bafd-34a347190566 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:58.280059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-efe5b64b-5110-4bb8-bafd-34a347190566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:58.289616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 952/1925] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:58.294774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5b64b-5110-4bb8-bafd-34a347190566 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:58.302308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60647cb1-8a00-4129-b13a-e5497c3a2474 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:58.302308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60647cb1-8a00-4129-b13a-e5497c3a2474 service glance] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:58.304263 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5b64b-5110-4bb8-bafd-34a347190566 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:58.306269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 974/1926] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:56:58.309231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8198ccf-2e91-4b86-a2c2-5eb65b0310c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:58.309844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198ccf-2e91-4b86-a2c2-5eb65b0310c1 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.310130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198ccf-2e91-4b86-a2c2-5eb65b0310c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.310408 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198ccf-2e91-4b86-a2c2-5eb65b0310c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.311138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8198ccf-2e91-4b86-a2c2-5eb65b0310c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:56:58.311446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8198ccf-2e91-4b86-a2c2-5eb65b0310c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:58.313345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8198ccf-2e91-4b86-a2c2-5eb65b0310c1 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:58.333718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 953/1927] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:56:58.337856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60647cb1-8a00-4129-b13a-e5497c3a2474 service glance] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:58.353009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60647cb1-8a00-4129-b13a-e5497c3a2474 service glance] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:58.361645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc67f0-c6c4-4ea1-9b92-2e3a5103b687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:58.362383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc67f0-c6c4-4ea1-9b92-2e3a5103b687 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.362673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc67f0-c6c4-4ea1-9b92-2e3a5103b687 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.362947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc67f0-c6c4-4ea1-9b92-2e3a5103b687 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.363711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc67f0-c6c4-4ea1-9b92-2e3a5103b687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:58.367568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc67f0-c6c4-4ea1-9b92-2e3a5103b687 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:58.445306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 975/1928] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:56:58 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:56:58.446441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8198ccf-2e91-4b86-a2c2-5eb65b0310c1 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:58.465019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8198ccf-2e91-4b86-a2c2-5eb65b0310c1 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:58.526033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149968d2-6ece-4e21-ac2c-df8ed97a05bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:58.527079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149968d2-6ece-4e21-ac2c-df8ed97a05bd admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.528085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149968d2-6ece-4e21-ac2c-df8ed97a05bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.528615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149968d2-6ece-4e21-ac2c-df8ed97a05bd admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.529554 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149968d2-6ece-4e21-ac2c-df8ed97a05bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:58.531847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149968d2-6ece-4e21-ac2c-df8ed97a05bd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:58.635178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 954/1929] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:56:58 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:56:58.636499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc67f0-c6c4-4ea1-9b92-2e3a5103b687 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:58.652918 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc67f0-c6c4-4ea1-9b92-2e3a5103b687 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:58.658490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94842a-f81e-44d2-a1a1-ff06c0a3edfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:58.659611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94842a-f81e-44d2-a1a1-ff06c0a3edfe admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.659611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94842a-f81e-44d2-a1a1-ff06c0a3edfe admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.659774 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94842a-f81e-44d2-a1a1-ff06c0a3edfe admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.660584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94842a-f81e-44d2-a1a1-ff06c0a3edfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:58.665791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94842a-f81e-44d2-a1a1-ff06c0a3edfe admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:58.677284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 976/1930] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:58.686162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149968d2-6ece-4e21-ac2c-df8ed97a05bd admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:58.697316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149968d2-6ece-4e21-ac2c-df8ed97a05bd admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:58.704553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909474ed-6686-40d9-b210-60689e20bfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:58.705597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909474ed-6686-40d9-b210-60689e20bfff admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.705713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909474ed-6686-40d9-b210-60689e20bfff admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.705984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909474ed-6686-40d9-b210-60689e20bfff admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.706713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909474ed-6686-40d9-b210-60689e20bfff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:58.708321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909474ed-6686-40d9-b210-60689e20bfff admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:58.723599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 955/1931] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:56:58.724998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94842a-f81e-44d2-a1a1-ff06c0a3edfe admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:58.736021 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94842a-f81e-44d2-a1a1-ff06c0a3edfe admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:58.740695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599a0a4-03ef-4394-8f76-ce47da36c7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:58.741581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a0a4-03ef-4394-8f76-ce47da36c7bf admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.741862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a0a4-03ef-4394-8f76-ce47da36c7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.742132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a0a4-03ef-4394-8f76-ce47da36c7bf admin admin] PATH_INFO: `/v3/projects/86963c999c1646f9b4f941a083bc8287/users/0356d99c53ec47788a4cdd632a20b6aa/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.757856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599a0a4-03ef-4394-8f76-ce47da36c7bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=86963c999c1646f9b4f941a083bc8287, user_id=0356d99c53ec47788a4cdd632a20b6aa, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:58.760464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599a0a4-03ef-4394-8f76-ce47da36c7bf admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:58.821147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:58.828283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909474ed-6686-40d9-b210-60689e20bfff admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:58.830644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-909474ed-6686-40d9-b210-60689e20bfff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:58.845709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909474ed-6686-40d9-b210-60689e20bfff admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:58.851003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:58.854741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.854741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.854741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] PATH_INFO: `/v3/projects/30385f3613104912a0f18f7e153950fe` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.854741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=30385f3613104912a0f18f7e153950fe)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:58.856639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:58.873133 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 977/1932] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:56:58 2022] PUT /identity/v3/projects/86963c999c1646f9b4f941a083bc8287/users/0356d99c53ec47788a4cdd632a20b6aa/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:58.884287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6920-a7a2-479d-aef4-58ac3a74c37b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:58.887166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6920-a7a2-479d-aef4-58ac3a74c37b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:58.888358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e6920-a7a2-479d-aef4-58ac3a74c37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:58.926317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f21e6920-a7a2-479d-aef4-58ac3a74c37b None None] MFA Rules not processed for user `0356d99c53ec47788a4cdd632a20b6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:58.927561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f21e6920-a7a2-479d-aef4-58ac3a74c37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:58.977098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:58.983823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30385f3613104912a0f18f7e153950fe', 'request_id': 'req-98bdffc1-aea3-43cf-806e-4887506e5a8f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.005538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 978/1933] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:56:59.056376 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e6920-a7a2-479d-aef4-58ac3a74c37b None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:59.057845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f21e6920-a7a2-479d-aef4-58ac3a74c37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:59.061376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30385f3613104912a0f18f7e153950fe', 'request_id': 'req-98bdffc1-aea3-43cf-806e-4887506e5a8f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.068141 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30385f3613104912a0f18f7e153950fe', 'request_id': 'req-98bdffc1-aea3-43cf-806e-4887506e5a8f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.080150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] The token cache is being invalidate because project 30385f3613104912a0f18f7e153950fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:56:59.080444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.081282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.083063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 956/1934] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:58 2022] DELETE /identity/v3/projects/30385f3613104912a0f18f7e153950fe => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:56:59.087938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:59.089266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:59.116361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e6920-a7a2-479d-aef4-58ac3a74c37b None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:59.152301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdffc1-aea3-43cf-806e-4887506e5a8f admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:59.157176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:59.158133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:59.158433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:59.158709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] PATH_INFO: `/v3/users/1800733a032d4f4fa773a787a4d7d91c` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:59.159909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1800733a032d4f4fa773a787a4d7d91c)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:59.161920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:59.164187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca1051-1697-45a5-a81a-26adf0d2c79a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:59.164833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca1051-1697-45a5-a81a-26adf0d2c79a service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:59.165115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca1051-1697-45a5-a81a-26adf0d2c79a service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:59.165434 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca1051-1697-45a5-a81a-26adf0d2c79a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:59.167238 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-99ca1051-1697-45a5-a81a-26adf0d2c79a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:59.246414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca1051-1697-45a5-a81a-26adf0d2c79a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:59.248381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca1051-1697-45a5-a81a-26adf0d2c79a service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:59.261719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 979/1935] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:56:59.263618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca1051-1697-45a5-a81a-26adf0d2c79a service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:59.266192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-99ca1051-1697-45a5-a81a-26adf0d2c79a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:59.281400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca1051-1697-45a5-a81a-26adf0d2c79a service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:59.286382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb6eec-b9a9-47b8-8130-834405f87bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:59.287254 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb6eec-b9a9-47b8-8130-834405f87bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:59.287757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb6eec-b9a9-47b8-8130-834405f87bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:59.288053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb6eec-b9a9-47b8-8130-834405f87bc2 admin admin] PATH_INFO: `/v3/users/f5a460d9b61443009f0dce391904b7f5` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:59.289292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb6eec-b9a9-47b8-8130-834405f87bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a460d9b61443009f0dce391904b7f5)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:59.290118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:59.291216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb6eec-b9a9-47b8-8130-834405f87bc2 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:59.361210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1800733a032d4f4fa773a787a4d7d91c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.368104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1800733a032d4f4fa773a787a4d7d91c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.368810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1800733a032d4f4fa773a787a4d7d91c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.381018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1800733a032d4f4fa773a787a4d7d91c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.381737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1800733a032d4f4fa773a787a4d7d91c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.478451 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 957/1936] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:59 2022] DELETE /identity/v3/users/1800733a032d4f4fa773a787a4d7d91c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:59.479582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:59.488232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-19fb6eec-b9a9-47b8-8130-834405f87bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a460d9b61443009f0dce391904b7f5'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.489223 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:59.489493 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-19fb6eec-b9a9-47b8-8130-834405f87bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a460d9b61443009f0dce391904b7f5'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.499226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-19fb6eec-b9a9-47b8-8130-834405f87bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a460d9b61443009f0dce391904b7f5'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.500678 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-19fb6eec-b9a9-47b8-8130-834405f87bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a460d9b61443009f0dce391904b7f5'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.517417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-19fb6eec-b9a9-47b8-8130-834405f87bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a460d9b61443009f0dce391904b7f5'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.562998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:59.567543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a93690-ccfa-48e8-bbc5-5ff4a84fd644 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:59.578058 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:59.579289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:59.579592 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:59.579860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] PATH_INFO: `/v3/users/0356d99c53ec47788a4cdd632a20b6aa` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:59.581204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0356d99c53ec47788a4cdd632a20b6aa)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:59.583345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:59.632037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 980/1937] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:59 2022] DELETE /identity/v3/users/f5a460d9b61443009f0dce391904b7f5 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:59.638205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb06f98-ef8b-443e-b127-f1abed799acf None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:59.638497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb06f98-ef8b-443e-b127-f1abed799acf None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:59.638778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb06f98-ef8b-443e-b127-f1abed799acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:59.651636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:56:59.657728 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9bb06f98-ef8b-443e-b127-f1abed799acf None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:59.658989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9bb06f98-ef8b-443e-b127-f1abed799acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:59.728881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 981/1938] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:59.736079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb06f98-ef8b-443e-b127-f1abed799acf None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:59.738115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9bb06f98-ef8b-443e-b127-f1abed799acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:59.776789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0356d99c53ec47788a4cdd632a20b6aa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.784983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0356d99c53ec47788a4cdd632a20b6aa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.792662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0356d99c53ec47788a4cdd632a20b6aa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.806037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0356d99c53ec47788a4cdd632a20b6aa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.806767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0356d99c53ec47788a4cdd632a20b6aa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:56:59.810937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb06f98-ef8b-443e-b127-f1abed799acf None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:56:59.814885 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d56ae-87bb-4af9-af3f-af05f18557ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:56:59.825344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d56ae-87bb-4af9-af3f-af05f18557ab service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:59.825344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d56ae-87bb-4af9-af3f-af05f18557ab service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:59.825344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d56ae-87bb-4af9-af3f-af05f18557ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:59.825344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f78d56ae-87bb-4af9-af3f-af05f18557ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:59.831665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d56ae-87bb-4af9-af3f-af05f18557ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:56:59.833206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d56ae-87bb-4af9-af3f-af05f18557ab service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:56:59.844938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 982/1939] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:56:59.877907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 958/1940] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:56:59 2022] DELETE /identity/v3/users/0356d99c53ec47788a4cdd632a20b6aa => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:56:59.882673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e6743-381c-4e6f-addf-9ae6460463a8 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:56:59.883056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e6743-381c-4e6f-addf-9ae6460463a8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:56:59.883413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e6743-381c-4e6f-addf-9ae6460463a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:56:59.902575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-732e6743-381c-4e6f-addf-9ae6460463a8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:56:59.904507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-732e6743-381c-4e6f-addf-9ae6460463a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:59.968545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 983/1941] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:56:59.978547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:59.980752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:56:59.993471 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e6743-381c-4e6f-addf-9ae6460463a8 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:56:59.995813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-732e6743-381c-4e6f-addf-9ae6460463a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:00.011630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e6743-381c-4e6f-addf-9ae6460463a8 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:00.025189 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:00.025189 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:00.025189 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:00.025189 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] PATH_INFO: `/v3/projects/a8c4b821f8834a5b8e6d5f8b678d83ab` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:00.026741 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c4b821f8834a5b8e6d5f8b678d83ab)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:00.029023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:00.072283 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:00.074010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8998fa3-730f-4301-957f-d4b8027dd256 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:00.078533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee044eb-345f-4a96-a0a5-eee1383f6b93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:00.079196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee044eb-345f-4a96-a0a5-eee1383f6b93 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:00.079485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee044eb-345f-4a96-a0a5-eee1383f6b93 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:00.079761 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee044eb-345f-4a96-a0a5-eee1383f6b93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:00.089440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee044eb-345f-4a96-a0a5-eee1383f6b93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:00.090906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee044eb-345f-4a96-a0a5-eee1383f6b93 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:00.100648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 959/1942] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:56:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:00.149189 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c4b821f8834a5b8e6d5f8b678d83ab', 'request_id': 'req-daa89d0f-73ce-4f53-9d86-a354ef111ced'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.236082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c4b821f8834a5b8e6d5f8b678d83ab', 'request_id': 'req-daa89d0f-73ce-4f53-9d86-a354ef111ced'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.243227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c4b821f8834a5b8e6d5f8b678d83ab', 'request_id': 'req-daa89d0f-73ce-4f53-9d86-a354ef111ced'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.250337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:00.255725 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] The token cache is being invalidate because project a8c4b821f8834a5b8e6d5f8b678d83ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:00.256094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.256773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-daa89d0f-73ce-4f53-9d86-a354ef111ced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.258368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 984/1943] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:56:59 2022] DELETE /identity/v3/projects/a8c4b821f8834a5b8e6d5f8b678d83ab => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:00.264915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93d535-b297-4918-908a-787126c1b4b3 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:00.265627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93d535-b297-4918-908a-787126c1b4b3 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:00.265724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93d535-b297-4918-908a-787126c1b4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:00.284489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ee93d535-b297-4918-908a-787126c1b4b3 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:00.285683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ee93d535-b297-4918-908a-787126c1b4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:00.390927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 960/1944] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:00.399125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b57f7d-c451-43a9-8833-f890a819dd39 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:00.399684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b57f7d-c451-43a9-8833-f890a819dd39 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:00.400088 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b57f7d-c451-43a9-8833-f890a819dd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:00.419636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-79b57f7d-c451-43a9-8833-f890a819dd39 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:00.420969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-79b57f7d-c451-43a9-8833-f890a819dd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:00.435304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 985/1945] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:00.437642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93d535-b297-4918-908a-787126c1b4b3 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:00.449104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93d535-b297-4918-908a-787126c1b4b3 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:00.455422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb51389-b5ff-48d4-a3b3-12fe62cacd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:00.456130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb51389-b5ff-48d4-a3b3-12fe62cacd5f admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:00.456388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb51389-b5ff-48d4-a3b3-12fe62cacd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:00.456671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb51389-b5ff-48d4-a3b3-12fe62cacd5f admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:00.457475 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb51389-b5ff-48d4-a3b3-12fe62cacd5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:00.457790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb51389-b5ff-48d4-a3b3-12fe62cacd5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:00.459444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb51389-b5ff-48d4-a3b3-12fe62cacd5f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:00.472431 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 961/1946] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:00.476934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b57f7d-c451-43a9-8833-f890a819dd39 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:00.488459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb51389-b5ff-48d4-a3b3-12fe62cacd5f admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:00.488781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b57f7d-c451-43a9-8833-f890a819dd39 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:00.490818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0eb51389-b5ff-48d4-a3b3-12fe62cacd5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:00.495644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d22a9-cbd8-4a16-9f03-91ee59f1dcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:00.496272 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d22a9-cbd8-4a16-9f03-91ee59f1dcb9 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:00.496562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d22a9-cbd8-4a16-9f03-91ee59f1dcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:00.496844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d22a9-cbd8-4a16-9f03-91ee59f1dcb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:00.497584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d22a9-cbd8-4a16-9f03-91ee59f1dcb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:00.501815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d22a9-cbd8-4a16-9f03-91ee59f1dcb9 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:00.523023 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb51389-b5ff-48d4-a3b3-12fe62cacd5f admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:00.528342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:00.529477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:00.529756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:00.530058 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] PATH_INFO: `/v3/projects/5f1be5bbf6de4b7d89bf438cd6a4d263` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:00.531212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f1be5bbf6de4b7d89bf438cd6a4d263)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:00.533145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:00.605834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 986/1947] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:00 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:00.605834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d22a9-cbd8-4a16-9f03-91ee59f1dcb9 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:00.605834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-914d22a9-cbd8-4a16-9f03-91ee59f1dcb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:00.664156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f1be5bbf6de4b7d89bf438cd6a4d263', 'request_id': 'req-0df7bb5a-d30e-44fc-a089-be58b39b7140'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.681979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d22a9-cbd8-4a16-9f03-91ee59f1dcb9 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:00.739456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:00.740323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:00.744244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:00.744244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] PATH_INFO: `/v3/projects/86963c999c1646f9b4f941a083bc8287` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:00.744244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86963c999c1646f9b4f941a083bc8287)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:00.744244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:00.763648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f1be5bbf6de4b7d89bf438cd6a4d263', 'request_id': 'req-0df7bb5a-d30e-44fc-a089-be58b39b7140'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.770742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f1be5bbf6de4b7d89bf438cd6a4d263', 'request_id': 'req-0df7bb5a-d30e-44fc-a089-be58b39b7140'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.783485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] The token cache is being invalidate because project 5f1be5bbf6de4b7d89bf438cd6a4d263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:00.783777 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.784735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.786364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 962/1948] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:00 2022] DELETE /identity/v3/projects/5f1be5bbf6de4b7d89bf438cd6a4d263 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:00.790180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:00.792493 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:00.805439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7bb5a-d30e-44fc-a089-be58b39b7140 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:00.810737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4af66c-f084-4b9e-ade2-5beb293c4ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:00.811463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4af66c-f084-4b9e-ade2-5beb293c4ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:00.811710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4af66c-f084-4b9e-ade2-5beb293c4ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:00.811991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4af66c-f084-4b9e-ade2-5beb293c4ba7 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:00.812685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4af66c-f084-4b9e-ade2-5beb293c4ba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:00.814666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4af66c-f084-4b9e-ade2-5beb293c4ba7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:00.860990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86963c999c1646f9b4f941a083bc8287', 'request_id': 'req-f1c525f4-8039-4874-83c5-2bc8ed4619b9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.918591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 963/1949] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:00 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:00.919521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4af66c-f084-4b9e-ade2-5beb293c4ba7 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:00.921084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1d4af66c-f084-4b9e-ade2-5beb293c4ba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:00.932745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4af66c-f084-4b9e-ade2-5beb293c4ba7 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:00.936294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86963c999c1646f9b4f941a083bc8287', 'request_id': 'req-f1c525f4-8039-4874-83c5-2bc8ed4619b9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.942460 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86963c999c1646f9b4f941a083bc8287', 'request_id': 'req-f1c525f4-8039-4874-83c5-2bc8ed4619b9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.959556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] The token cache is being invalidate because project 86963c999c1646f9b4f941a083bc8287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:00.960709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.962563 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:00.964988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 987/1950] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:00 2022] DELETE /identity/v3/projects/86963c999c1646f9b4f941a083bc8287 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:00.967492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:00.969380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:00.993762 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:00.994649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:00.994941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:00.995214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] PATH_INFO: `/v3/users/63a80eaf89fe4fd7b741c18779674943` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:00.997008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a80eaf89fe4fd7b741c18779674943)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:00.998945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.022110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:01.037458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c525f4-8039-4874-83c5-2bc8ed4619b9 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.042295 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32144d8b-576b-442b-8486-f6c860c7a4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.043000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32144d8b-576b-442b-8486-f6c860c7a4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.043356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32144d8b-576b-442b-8486-f6c860c7a4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.043637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32144d8b-576b-442b-8486-f6c860c7a4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.044359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32144d8b-576b-442b-8486-f6c860c7a4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.045992 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32144d8b-576b-442b-8486-f6c860c7a4d0 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.055845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 988/1951] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:01.057023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb386d8-1bf4-49f1-ab58-3a227a53e216 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.057335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb386d8-1bf4-49f1-ab58-3a227a53e216 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.057619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb386d8-1bf4-49f1-ab58-3a227a53e216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.075999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4bb386d8-1bf4-49f1-ab58-3a227a53e216 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:01.077216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4bb386d8-1bf4-49f1-ab58-3a227a53e216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:01.085215 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 989/1952] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:01.085997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb386d8-1bf4-49f1-ab58-3a227a53e216 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:01.095015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb386d8-1bf4-49f1-ab58-3a227a53e216 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.099675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f56cf5-748e-4da0-b274-9ad218d182b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.100303 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f56cf5-748e-4da0-b274-9ad218d182b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.101381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f56cf5-748e-4da0-b274-9ad218d182b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.101381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f56cf5-748e-4da0-b274-9ad218d182b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.101759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f56cf5-748e-4da0-b274-9ad218d182b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.103358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f56cf5-748e-4da0-b274-9ad218d182b0 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.115258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 990/1953] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:01.116837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53f721-cdbc-4030-8567-a474e7e0b11a None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.117135 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53f721-cdbc-4030-8567-a474e7e0b11a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.117530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53f721-cdbc-4030-8567-a474e7e0b11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.133577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a80eaf89fe4fd7b741c18779674943'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:01.138226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-df53f721-cdbc-4030-8567-a474e7e0b11a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:01.139464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-df53f721-cdbc-4030-8567-a474e7e0b11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:01.140432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a80eaf89fe4fd7b741c18779674943'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:01.141105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a80eaf89fe4fd7b741c18779674943'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:01.147412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 991/1954] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:01.148159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53f721-cdbc-4030-8567-a474e7e0b11a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:01.149416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-df53f721-cdbc-4030-8567-a474e7e0b11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:01.153219 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a80eaf89fe4fd7b741c18779674943'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:01.153943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a80eaf89fe4fd7b741c18779674943'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:01.160985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53f721-cdbc-4030-8567-a474e7e0b11a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.165738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfaa4d-5462-4d3f-8b12-ed6ad9600820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.166609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfaa4d-5462-4d3f-8b12-ed6ad9600820 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.166898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfaa4d-5462-4d3f-8b12-ed6ad9600820 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.167188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfaa4d-5462-4d3f-8b12-ed6ad9600820 admin admin] PATH_INFO: `/v3/projects/65c2153f8cc54c4192ce4d2c97681dfe/users/783e942c0e3f413e814308971b069245/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.169644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:01.182523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfaa4d-5462-4d3f-8b12-ed6ad9600820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c2153f8cc54c4192ce4d2c97681dfe, user_id=783e942c0e3f413e814308971b069245, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.186012 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfaa4d-5462-4d3f-8b12-ed6ad9600820 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.243056 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 964/1955] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:00 2022] DELETE /identity/v3/users/63a80eaf89fe4fd7b741c18779674943 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:01.244055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:01.311780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b35cc0-4071-4bce-9f2d-16c6dab97362 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.316522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d672b0-54d1-4a4c-8da2-62a116d51cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.317190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d672b0-54d1-4a4c-8da2-62a116d51cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.317509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d672b0-54d1-4a4c-8da2-62a116d51cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.317788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d672b0-54d1-4a4c-8da2-62a116d51cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.318503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d672b0-54d1-4a4c-8da2-62a116d51cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:01.318812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d672b0-54d1-4a4c-8da2-62a116d51cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.320446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d672b0-54d1-4a4c-8da2-62a116d51cc3 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.335001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 965/1956] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:01.339296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d672b0-54d1-4a4c-8da2-62a116d51cc3 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:01.352128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 992/1957] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:01 2022] PUT /identity/v3/projects/65c2153f8cc54c4192ce4d2c97681dfe/users/783e942c0e3f413e814308971b069245/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:01.356397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b3f1c-6af4-40af-a7f6-ab388e70eabb None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.356552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b3f1c-6af4-40af-a7f6-ab388e70eabb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.356886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b3f1c-6af4-40af-a7f6-ab388e70eabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.363521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d672b0-54d1-4a4c-8da2-62a116d51cc3 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.391308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-7a9b3f1c-6af4-40af-a7f6-ab388e70eabb None None] MFA Rules not processed for user `783e942c0e3f413e814308971b069245`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:01.392928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7a9b3f1c-6af4-40af-a7f6-ab388e70eabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:01.424737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501d1bc-f68f-49a2-b125-9d1a707f8eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.425440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501d1bc-f68f-49a2-b125-9d1a707f8eaf admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.425730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501d1bc-f68f-49a2-b125-9d1a707f8eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.426002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501d1bc-f68f-49a2-b125-9d1a707f8eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.426851 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501d1bc-f68f-49a2-b125-9d1a707f8eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.428748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501d1bc-f68f-49a2-b125-9d1a707f8eaf admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.461943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 993/1958] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:01.471294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b3f1c-6af4-40af-a7f6-ab388e70eabb None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:01.485587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b3f1c-6af4-40af-a7f6-ab388e70eabb None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.491744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed387e-41e2-48cf-8c88-2c0958bcd49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.492707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed387e-41e2-48cf-8c88-2c0958bcd49a admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.493066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed387e-41e2-48cf-8c88-2c0958bcd49a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.493432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed387e-41e2-48cf-8c88-2c0958bcd49a admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.494136 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed387e-41e2-48cf-8c88-2c0958bcd49a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.496110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed387e-41e2-48cf-8c88-2c0958bcd49a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.542656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 966/1959] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:01 2022] POST /identity/v3/projects => generated 391 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:01.546814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501d1bc-f68f-49a2-b125-9d1a707f8eaf admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:01.583780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 994/1960] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:01 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:01.590020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed387e-41e2-48cf-8c88-2c0958bcd49a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:01.616833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501d1bc-f68f-49a2-b125-9d1a707f8eaf admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.659456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed387e-41e2-48cf-8c88-2c0958bcd49a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.666521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c6778-12fc-45b4-93c3-bdecbad4b349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.667211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c6778-12fc-45b4-93c3-bdecbad4b349 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.667584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c6778-12fc-45b4-93c3-bdecbad4b349 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.673110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c6778-12fc-45b4-93c3-bdecbad4b349 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.673110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c6778-12fc-45b4-93c3-bdecbad4b349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.673110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c6778-12fc-45b4-93c3-bdecbad4b349 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.678415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c2b0c-5e03-4c04-8164-4225b5d3dc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.679119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c2b0c-5e03-4c04-8164-4225b5d3dc2a admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.679397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c2b0c-5e03-4c04-8164-4225b5d3dc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.679859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c2b0c-5e03-4c04-8164-4225b5d3dc2a admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.680589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c2b0c-5e03-4c04-8164-4225b5d3dc2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.682586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c2b0c-5e03-4c04-8164-4225b5d3dc2a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.787419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 995/1961] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:01 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:01.787643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c6778-12fc-45b4-93c3-bdecbad4b349 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:01.789223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-695c6778-12fc-45b4-93c3-bdecbad4b349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:01.801198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c6778-12fc-45b4-93c3-bdecbad4b349 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.806705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.807581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.807991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.812742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] PATH_INFO: `/v3/projects/d9ca02a8cde444b982cc18694fb00af6` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.812742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ca02a8cde444b982cc18694fb00af6)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.814066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.821096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 967/1962] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:01 2022] POST /identity/v3/users => generated 296 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:01.821096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c2b0c-5e03-4c04-8164-4225b5d3dc2a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:01.835405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c2b0c-5e03-4c04-8164-4225b5d3dc2a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.843190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6d9d4-2d4c-4620-a5d8-48c09aed87a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.843935 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6d9d4-2d4c-4620-a5d8-48c09aed87a6 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.844222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6d9d4-2d4c-4620-a5d8-48c09aed87a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.844494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d6d9d4-2d4c-4620-a5d8-48c09aed87a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.845226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6d9d4-2d4c-4620-a5d8-48c09aed87a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.847116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d6d9d4-2d4c-4620-a5d8-48c09aed87a6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.861435 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 968/1963] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:01.862328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6d9d4-2d4c-4620-a5d8-48c09aed87a6 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:01.874284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d6d9d4-2d4c-4620-a5d8-48c09aed87a6 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.880036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965849b-fab6-47f3-9975-ae39c2be9836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.880692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965849b-fab6-47f3-9975-ae39c2be9836 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.880978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965849b-fab6-47f3-9975-ae39c2be9836 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.881347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965849b-fab6-47f3-9975-ae39c2be9836 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.881985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965849b-fab6-47f3-9975-ae39c2be9836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.883592 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965849b-fab6-47f3-9975-ae39c2be9836 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:01.898882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 969/1964] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:01.898882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965849b-fab6-47f3-9975-ae39c2be9836 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:01.912719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965849b-fab6-47f3-9975-ae39c2be9836 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:01.925282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:01.936221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2ddb1-96c7-4865-bdb3-1b674ef591c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:01.937222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2ddb1-96c7-4865-bdb3-1b674ef591c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:01.937540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2ddb1-96c7-4865-bdb3-1b674ef591c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:01.937918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2ddb1-96c7-4865-bdb3-1b674ef591c8 admin admin] PATH_INFO: `/v3/projects/b59692d38c6544fcbc099e9d3f89e0ed/users/06077abaaa2241d3b07ef403506f1d49/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:01.949127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ca02a8cde444b982cc18694fb00af6', 'request_id': 'req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:01.956924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2ddb1-96c7-4865-bdb3-1b674ef591c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b59692d38c6544fcbc099e9d3f89e0ed, user_id=06077abaaa2241d3b07ef403506f1d49, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:01.959979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e2ddb1-96c7-4865-bdb3-1b674ef591c8 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.038495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ca02a8cde444b982cc18694fb00af6', 'request_id': 'req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:02.046398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ca02a8cde444b982cc18694fb00af6', 'request_id': 'req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:02.060197 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] The token cache is being invalidate because project d9ca02a8cde444b982cc18694fb00af6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:02.060668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:02.061532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:02.062867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 970/1965] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:01 2022] PUT /identity/v3/projects/b59692d38c6544fcbc099e9d3f89e0ed/users/06077abaaa2241d3b07ef403506f1d49/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:02.065845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 996/1966] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:01 2022] DELETE /identity/v3/projects/d9ca02a8cde444b982cc18694fb00af6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:02.065959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2ddb1-96c7-4865-bdb3-1b674ef591c8 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.065959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-54e2ddb1-96c7-4865-bdb3-1b674ef591c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:02.067110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.068958 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:02.131946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:INSERT=5,keystone:DELETE=13,keystone:SELECT=82 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:02.136867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2ddb1-96c7-4865-bdb3-1b674ef591c8 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.140023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a878fd1-6cb9-4838-9c46-f95d681c3bdf admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.141771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b60c1cb-b99b-4f39-a8e6-a32e7fca9762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.142438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60c1cb-b99b-4f39-a8e6-a32e7fca9762 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.142720 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60c1cb-b99b-4f39-a8e6-a32e7fca9762 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.143070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b60c1cb-b99b-4f39-a8e6-a32e7fca9762 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.143860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60c1cb-b99b-4f39-a8e6-a32e7fca9762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.144851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cacf62-305f-4953-b639-bbad0d61223a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.145490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b60c1cb-b99b-4f39-a8e6-a32e7fca9762 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.145590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cacf62-305f-4953-b639-bbad0d61223a admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.145830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cacf62-305f-4953-b639-bbad0d61223a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.146107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cacf62-305f-4953-b639-bbad0d61223a admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.146835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cacf62-305f-4953-b639-bbad0d61223a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.148451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cacf62-305f-4953-b639-bbad0d61223a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.155720 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 971/1967] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:02.157016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e43d87-7a3d-4879-9a87-817392168dfb None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.157370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e43d87-7a3d-4879-9a87-817392168dfb None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.157837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e43d87-7a3d-4879-9a87-817392168dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.158846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 997/1968] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:02.162641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cacf62-305f-4953-b639-bbad0d61223a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.175002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cacf62-305f-4953-b639-bbad0d61223a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.176512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-22e43d87-7a3d-4879-9a87-817392168dfb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:02.177271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6730d1-0555-4a8b-867e-9e352f209274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.177752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-22e43d87-7a3d-4879-9a87-817392168dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:02.178145 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6730d1-0555-4a8b-867e-9e352f209274 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.178421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6730d1-0555-4a8b-867e-9e352f209274 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.178742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6730d1-0555-4a8b-867e-9e352f209274 admin admin] PATH_INFO: `/v3/projects/0fcad64e35874948a0dd78fb42ce9404/users/c40751576420451c9fba2aebcffaf2f4/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.191356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 972/1969] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:02.192384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e43d87-7a3d-4879-9a87-817392168dfb None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.198356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6730d1-0555-4a8b-867e-9e352f209274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fcad64e35874948a0dd78fb42ce9404, user_id=c40751576420451c9fba2aebcffaf2f4, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.201091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6730d1-0555-4a8b-867e-9e352f209274 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.202896 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e43d87-7a3d-4879-9a87-817392168dfb None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.207820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9424967-98b0-433c-8d54-b7480376f16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.208705 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9424967-98b0-433c-8d54-b7480376f16d admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.208995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9424967-98b0-433c-8d54-b7480376f16d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.209299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9424967-98b0-433c-8d54-b7480376f16d admin admin] PATH_INFO: `/v3/projects/b59692d38c6544fcbc099e9d3f89e0ed/users/06077abaaa2241d3b07ef403506f1d49/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.211267 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9424967-98b0-433c-8d54-b7480376f16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b59692d38c6544fcbc099e9d3f89e0ed, user_id=06077abaaa2241d3b07ef403506f1d49, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.214027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9424967-98b0-433c-8d54-b7480376f16d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.224107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 973/1970] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:02 2022] PUT /identity/v3/projects/b59692d38c6544fcbc099e9d3f89e0ed/users/06077abaaa2241d3b07ef403506f1d49/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:02.225342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34d995-fda1-4151-bc51-358bcab7419e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.225631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34d995-fda1-4151-bc51-358bcab7419e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.225909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34d995-fda1-4151-bc51-358bcab7419e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.244359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-4c34d995-fda1-4151-bc51-358bcab7419e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:02.245595 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-4c34d995-fda1-4151-bc51-358bcab7419e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:02.295038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 998/1971] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:02 2022] PUT /identity/v3/projects/0fcad64e35874948a0dd78fb42ce9404/users/c40751576420451c9fba2aebcffaf2f4/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:02.296392 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2f9de-ee85-494b-89d4-d77dc3ed5c49 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.296681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2f9de-ee85-494b-89d4-d77dc3ed5c49 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.296957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2f9de-ee85-494b-89d4-d77dc3ed5c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.307961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 974/1972] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:02.309984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fb79-8a41-4e03-9f1f-67d4f17bbf15 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.309984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fb79-8a41-4e03-9f1f-67d4f17bbf15 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.310287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3fb79-8a41-4e03-9f1f-67d4f17bbf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.329045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-b6e2f9de-ee85-494b-89d4-d77dc3ed5c49 None None] MFA Rules not processed for user `06077abaaa2241d3b07ef403506f1d49`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:02.330302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b6e2f9de-ee85-494b-89d4-d77dc3ed5c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:02.337572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-49f3fb79-8a41-4e03-9f1f-67d4f17bbf15 None None] MFA Rules not processed for user `c40751576420451c9fba2aebcffaf2f4`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:02.338803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-49f3fb79-8a41-4e03-9f1f-67d4f17bbf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:02.405530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 999/1973] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:02.406759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2f9de-ee85-494b-89d4-d77dc3ed5c49 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.410038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 975/1974] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:02.416275 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3fb79-8a41-4e03-9f1f-67d4f17bbf15 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.417870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-49f3fb79-8a41-4e03-9f1f-67d4f17bbf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:02.476796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:02.481037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2f9de-ee85-494b-89d4-d77dc3ed5c49 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.485128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3fb79-8a41-4e03-9f1f-67d4f17bbf15 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.485724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f17eb-f922-487d-b9a3-7b3c3c640b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.486427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f17eb-f922-487d-b9a3-7b3c3c640b61 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.486747 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f17eb-f922-487d-b9a3-7b3c3c640b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.487026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f17eb-f922-487d-b9a3-7b3c3c640b61 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.487753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f17eb-f922-487d-b9a3-7b3c3c640b61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:02.488064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f17eb-f922-487d-b9a3-7b3c3c640b61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.488988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a425df-908d-4648-b8c2-13ab1d809f50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.489665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a425df-908d-4648-b8c2-13ab1d809f50 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.489952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a425df-908d-4648-b8c2-13ab1d809f50 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.490225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a425df-908d-4648-b8c2-13ab1d809f50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.490322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f17eb-f922-487d-b9a3-7b3c3c640b61 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.494944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:02.501620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a425df-908d-4648-b8c2-13ab1d809f50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.505428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1000/1975] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:02.505428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f17eb-f922-487d-b9a3-7b3c3c640b61 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.505751 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a425df-908d-4648-b8c2-13ab1d809f50 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.507064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-742f17eb-f922-487d-b9a3-7b3c3c640b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:02.515161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 976/1976] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:02 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:57:02.516324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a425df-908d-4648-b8c2-13ab1d809f50 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.527725 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a425df-908d-4648-b8c2-13ab1d809f50 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.535974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69753f57-4ea9-4f0a-a3d6-aceba1f44750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.536091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f17eb-f922-487d-b9a3-7b3c3c640b61 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.536195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69753f57-4ea9-4f0a-a3d6-aceba1f44750 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.536195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69753f57-4ea9-4f0a-a3d6-aceba1f44750 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.536195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69753f57-4ea9-4f0a-a3d6-aceba1f44750 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.536497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69753f57-4ea9-4f0a-a3d6-aceba1f44750 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.538506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69753f57-4ea9-4f0a-a3d6-aceba1f44750 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.545864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587e7f5-0e7c-440c-8734-f12eefe1ff7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.546602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587e7f5-0e7c-440c-8734-f12eefe1ff7e service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.546954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587e7f5-0e7c-440c-8734-f12eefe1ff7e service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.547335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587e7f5-0e7c-440c-8734-f12eefe1ff7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.556485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587e7f5-0e7c-440c-8734-f12eefe1ff7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.558171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587e7f5-0e7c-440c-8734-f12eefe1ff7e service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.568331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1001/1977] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:02.581505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:02.646718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 977/1978] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:02 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:02.647472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587e7f5-0e7c-440c-8734-f12eefe1ff7e service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.656346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:02.659296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69753f57-4ea9-4f0a-a3d6-aceba1f44750 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.664441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587e7f5-0e7c-440c-8734-f12eefe1ff7e service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.719579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69753f57-4ea9-4f0a-a3d6-aceba1f44750 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.723347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012d9a4-dcc1-4b71-86d5-80576f3e5305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.724000 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012d9a4-dcc1-4b71-86d5-80576f3e5305 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.724556 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012d9a4-dcc1-4b71-86d5-80576f3e5305 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.724843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012d9a4-dcc1-4b71-86d5-80576f3e5305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.726789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6012d9a4-dcc1-4b71-86d5-80576f3e5305 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:02.728401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afc06-30d6-478f-91ca-ce6d1c38b86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.729541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afc06-30d6-478f-91ca-ce6d1c38b86d admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.729899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afc06-30d6-478f-91ca-ce6d1c38b86d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.731313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54afc06-30d6-478f-91ca-ce6d1c38b86d admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.732265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54afc06-30d6-478f-91ca-ce6d1c38b86d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.734553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54afc06-30d6-478f-91ca-ce6d1c38b86d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.787292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012d9a4-dcc1-4b71-86d5-80576f3e5305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.788724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012d9a4-dcc1-4b71-86d5-80576f3e5305 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.797537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 978/1979] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:02.852372 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1002/1980] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:02 2022] POST /identity/v3/users => generated 284 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:02.853616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012d9a4-dcc1-4b71-86d5-80576f3e5305 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.864593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012d9a4-dcc1-4b71-86d5-80576f3e5305 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.869291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c912f-da90-4b8d-932b-5122523b6bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.869955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c912f-da90-4b8d-932b-5122523b6bda admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.870243 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c912f-da90-4b8d-932b-5122523b6bda admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.876423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c912f-da90-4b8d-932b-5122523b6bda admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.876423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c912f-da90-4b8d-932b-5122523b6bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.876423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c912f-da90-4b8d-932b-5122523b6bda admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.882757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 979/1981] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:02.886760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afc06-30d6-478f-91ca-ce6d1c38b86d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.897445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54afc06-30d6-478f-91ca-ce6d1c38b86d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.902487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6275e6c3-7b0c-4832-aa4c-e1d4c67a0332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.903757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275e6c3-7b0c-4832-aa4c-e1d4c67a0332 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.904040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275e6c3-7b0c-4832-aa4c-e1d4c67a0332 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.904966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6275e6c3-7b0c-4832-aa4c-e1d4c67a0332 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.905062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275e6c3-7b0c-4832-aa4c-e1d4c67a0332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.906669 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6275e6c3-7b0c-4832-aa4c-e1d4c67a0332 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:02.918022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1003/1982] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:02.922184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c912f-da90-4b8d-932b-5122523b6bda admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:02.932643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c912f-da90-4b8d-932b-5122523b6bda admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:02.939736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45076cdf-38c6-4be9-bfae-96c87983950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:02.940643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45076cdf-38c6-4be9-bfae-96c87983950b admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:02.940925 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45076cdf-38c6-4be9-bfae-96c87983950b admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:02.941210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45076cdf-38c6-4be9-bfae-96c87983950b admin admin] PATH_INFO: `/v3/projects/c382d42027224ce8807082329d6a0b1d/users/556d834afd0d49129538d5a28809dcae/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:02.958526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45076cdf-38c6-4be9-bfae-96c87983950b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c382d42027224ce8807082329d6a0b1d, user_id=556d834afd0d49129538d5a28809dcae, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:02.961571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45076cdf-38c6-4be9-bfae-96c87983950b admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:03.004155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:03.037658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 980/1983] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:02 2022] PUT /identity/v3/projects/c382d42027224ce8807082329d6a0b1d/users/556d834afd0d49129538d5a28809dcae/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:03.048898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7315b-f583-4b01-a857-7b9c765e4ad6 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:03.049324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7315b-f583-4b01-a857-7b9c765e4ad6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:03.049680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7315b-f583-4b01-a857-7b9c765e4ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:03.077350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-94b7315b-f583-4b01-a857-7b9c765e4ad6 None None] MFA Rules not processed for user `556d834afd0d49129538d5a28809dcae`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:03.078540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-94b7315b-f583-4b01-a857-7b9c765e4ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:03.121870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:03.157954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1004/1984] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:03.162429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45076cdf-38c6-4be9-bfae-96c87983950b admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:03.228257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45076cdf-38c6-4be9-bfae-96c87983950b admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:03.233669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb68f2-da9b-4468-af47-af44f229e9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:03.234361 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb68f2-da9b-4468-af47-af44f229e9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:03.238612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb68f2-da9b-4468-af47-af44f229e9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:03.238612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb68f2-da9b-4468-af47-af44f229e9d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:03.238612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb68f2-da9b-4468-af47-af44f229e9d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:03.238612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb68f2-da9b-4468-af47-af44f229e9d1 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:03.406380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 981/1985] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:03 2022] POST /identity/v3/projects => generated 367 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:03.411541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7315b-f583-4b01-a857-7b9c765e4ad6 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:03.422670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7315b-f583-4b01-a857-7b9c765e4ad6 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:03.486571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62766c3-1f97-4160-af78-7340b7a20625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:03.487314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62766c3-1f97-4160-af78-7340b7a20625 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:03.487601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62766c3-1f97-4160-af78-7340b7a20625 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:03.487877 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62766c3-1f97-4160-af78-7340b7a20625 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:03.488628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62766c3-1f97-4160-af78-7340b7a20625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:03.490683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62766c3-1f97-4160-af78-7340b7a20625 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:03.594652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1005/1986] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:03 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:03.598440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb68f2-da9b-4468-af47-af44f229e9d1 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:03.614855 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb68f2-da9b-4468-af47-af44f229e9d1 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:03.631587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a9e30-d9c2-4d4f-a6bc-c9547898517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:03.632629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a9e30-d9c2-4d4f-a6bc-c9547898517c admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:03.633065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a9e30-d9c2-4d4f-a6bc-c9547898517c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:03.633767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a9e30-d9c2-4d4f-a6bc-c9547898517c admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:03.634943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a9e30-d9c2-4d4f-a6bc-c9547898517c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:03.637096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a9e30-d9c2-4d4f-a6bc-c9547898517c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:03.650427 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 982/1987] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:03.657589 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62766c3-1f97-4160-af78-7340b7a20625 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:03.670629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62766c3-1f97-4160-af78-7340b7a20625 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:03.677409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd3616-8f25-458b-8692-d91f4b4f2553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:03.678360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3616-8f25-458b-8692-d91f4b4f2553 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:03.678610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3616-8f25-458b-8692-d91f4b4f2553 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:03.679046 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3616-8f25-458b-8692-d91f4b4f2553 admin admin] PATH_INFO: `/v3/projects/01b10169cfd946088945f65ab9e37ad3/users/e771400f43074aeebfa4ceac7f962ed0/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:03.699984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd3616-8f25-458b-8692-d91f4b4f2553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b10169cfd946088945f65ab9e37ad3, user_id=e771400f43074aeebfa4ceac7f962ed0, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:03.702650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd3616-8f25-458b-8692-d91f4b4f2553 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:03.781550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1006/1988] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:03 2022] PUT /identity/v3/projects/01b10169cfd946088945f65ab9e37ad3/users/e771400f43074aeebfa4ceac7f962ed0/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:03.785528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a9e30-d9c2-4d4f-a6bc-c9547898517c admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:03.796207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a9e30-d9c2-4d4f-a6bc-c9547898517c admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:03.850540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d0378-af2b-4b59-88bd-7802a5c8aee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:03.851128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d0378-af2b-4b59-88bd-7802a5c8aee8 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:03.851410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d0378-af2b-4b59-88bd-7802a5c8aee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:03.851687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d0378-af2b-4b59-88bd-7802a5c8aee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:03.852389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d0378-af2b-4b59-88bd-7802a5c8aee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:03.854030 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d0378-af2b-4b59-88bd-7802a5c8aee8 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:03.864693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 983/1989] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:03.868869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd3616-8f25-458b-8692-d91f4b4f2553 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:03.879231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd3616-8f25-458b-8692-d91f4b4f2553 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:03.883872 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a971b3-5348-478a-8f48-5f89f735f78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:03.884748 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a971b3-5348-478a-8f48-5f89f735f78e admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:03.885040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a971b3-5348-478a-8f48-5f89f735f78e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:03.885354 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a971b3-5348-478a-8f48-5f89f735f78e admin admin] PATH_INFO: `/v3/projects/01b10169cfd946088945f65ab9e37ad3/users/e771400f43074aeebfa4ceac7f962ed0/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:03.887332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a971b3-5348-478a-8f48-5f89f735f78e admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b10169cfd946088945f65ab9e37ad3, user_id=e771400f43074aeebfa4ceac7f962ed0, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:03.890017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a971b3-5348-478a-8f48-5f89f735f78e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:03.900216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1007/1990] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:03 2022] PUT /identity/v3/projects/01b10169cfd946088945f65ab9e37ad3/users/e771400f43074aeebfa4ceac7f962ed0/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:03.910766 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e384c-57e5-4437-91d1-ff61157aa44b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:03.911066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e384c-57e5-4437-91d1-ff61157aa44b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:03.911353 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e384c-57e5-4437-91d1-ff61157aa44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:03.938906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2c6e384c-57e5-4437-91d1-ff61157aa44b None None] MFA Rules not processed for user `e771400f43074aeebfa4ceac7f962ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:03.940311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2c6e384c-57e5-4437-91d1-ff61157aa44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:03.954481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:04.009887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 984/1991] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:04.022155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a971b3-5348-478a-8f48-5f89f735f78e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:04.078106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a971b3-5348-478a-8f48-5f89f735f78e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:04.081793 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dcdaa-1d60-453b-8a97-0c2ca14bc2bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:04.082436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dcdaa-1d60-453b-8a97-0c2ca14bc2bd service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:04.082723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dcdaa-1d60-453b-8a97-0c2ca14bc2bd service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:04.083000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dcdaa-1d60-453b-8a97-0c2ca14bc2bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:04.091827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069dcdaa-1d60-453b-8a97-0c2ca14bc2bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:04.093287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069dcdaa-1d60-453b-8a97-0c2ca14bc2bd service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:04.102570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1008/1992] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:04 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:04.242825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e384c-57e5-4437-91d1-ff61157aa44b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:04.253317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e384c-57e5-4437-91d1-ff61157aa44b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:04.258261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c9277-b4b1-4f8f-90f4-0b7c29fdae60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:04.258261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c9277-b4b1-4f8f-90f4-0b7c29fdae60 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:04.258261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c9277-b4b1-4f8f-90f4-0b7c29fdae60 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:04.258261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c9277-b4b1-4f8f-90f4-0b7c29fdae60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:04.316311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c9277-b4b1-4f8f-90f4-0b7c29fdae60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:04.317789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c9277-b4b1-4f8f-90f4-0b7c29fdae60 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:04.329926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 985/1993] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:09.270745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dcdaa-1d60-453b-8a97-0c2ca14bc2bd service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:09.307490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:09.354407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dcdaa-1d60-453b-8a97-0c2ca14bc2bd service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:09.359967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96bb9e-9335-4c53-8028-f3f83244b9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:09.360958 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bb9e-9335-4c53-8028-f3f83244b9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:09.361123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bb9e-9335-4c53-8028-f3f83244b9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:09.361814 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bb9e-9335-4c53-8028-f3f83244b9e7 admin admin] PATH_INFO: `/v3/users/556d834afd0d49129538d5a28809dcae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:09.362756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96bb9e-9335-4c53-8028-f3f83244b9e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=556d834afd0d49129538d5a28809dcae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:09.364717 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96bb9e-9335-4c53-8028-f3f83244b9e7 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:09.578521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ad96bb9e-9335-4c53-8028-f3f83244b9e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '556d834afd0d49129538d5a28809dcae'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:09.579244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ad96bb9e-9335-4c53-8028-f3f83244b9e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '556d834afd0d49129538d5a28809dcae'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:09.586866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ad96bb9e-9335-4c53-8028-f3f83244b9e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '556d834afd0d49129538d5a28809dcae'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:09.587499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ad96bb9e-9335-4c53-8028-f3f83244b9e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '556d834afd0d49129538d5a28809dcae'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:09.600303 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ad96bb9e-9335-4c53-8028-f3f83244b9e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '556d834afd0d49129538d5a28809dcae'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:09.648976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c9277-b4b1-4f8f-90f4-0b7c29fdae60 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:09.662286 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c9277-b4b1-4f8f-90f4-0b7c29fdae60 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:09.667294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:09.668811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:09.669154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:09.669520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] PATH_INFO: `/v3/users/783e942c0e3f413e814308971b069245` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:09.680794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1009/1994] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:09 2022] DELETE /identity/v3/users/556d834afd0d49129538d5a28809dcae => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:09.681033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] RBAC: Authorizing `identity:delete_user(user_id=783e942c0e3f413e814308971b069245)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:09.682909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa8186-b200-4cd0-9dd8-7937118990bb None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:09.683265 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa8186-b200-4cd0-9dd8-7937118990bb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:09.684859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa8186-b200-4cd0-9dd8-7937118990bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:09.694160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:09.714692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-59aa8186-b200-4cd0-9dd8-7937118990bb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:09.716876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-59aa8186-b200-4cd0-9dd8-7937118990bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:09.802087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1010/1995] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:09.814557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa8186-b200-4cd0-9dd8-7937118990bb None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:09.816250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-59aa8186-b200-4cd0-9dd8-7937118990bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:09.877560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa8186-b200-4cd0-9dd8-7937118990bb None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:09.881150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d4e1d-5e04-49a7-9abd-5938e253d4e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:09.881827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d4e1d-5e04-49a7-9abd-5938e253d4e1 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:09.882110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d4e1d-5e04-49a7-9abd-5938e253d4e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:09.882386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d4e1d-5e04-49a7-9abd-5938e253d4e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:09.888774 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:09.892593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d4e1d-5e04-49a7-9abd-5938e253d4e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:09.894167 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61d4e1d-5e04-49a7-9abd-5938e253d4e1 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:09.905703 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1011/1996] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:57:09.928915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '783e942c0e3f413e814308971b069245'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:09.938199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '783e942c0e3f413e814308971b069245'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:09.946869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '783e942c0e3f413e814308971b069245'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:09.968461 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '783e942c0e3f413e814308971b069245'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:09.969891 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '783e942c0e3f413e814308971b069245'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:10.049218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 986/1997] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:09 2022] DELETE /identity/v3/users/783e942c0e3f413e814308971b069245 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:10.053813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831d0e5-1d21-4561-8d02-1cf1e64d2e8f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:10.054181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831d0e5-1d21-4561-8d02-1cf1e64d2e8f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:10.054465 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4831d0e5-1d21-4561-8d02-1cf1e64d2e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:10.072447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4831d0e5-1d21-4561-8d02-1cf1e64d2e8f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:10.074603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4831d0e5-1d21-4561-8d02-1cf1e64d2e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:10.147076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1012/1998] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:10.162882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:10.164414 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:10.193874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:10.226432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bf758-c672-4770-b5bd-9c774611b55f admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:10.240415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e86b5-b80e-43db-a4c3-76e0cf632a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:10.241332 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e86b5-b80e-43db-a4c3-76e0cf632a2b service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:10.241535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e86b5-b80e-43db-a4c3-76e0cf632a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:10.241799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e86b5-b80e-43db-a4c3-76e0cf632a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:10.257007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e86b5-b80e-43db-a4c3-76e0cf632a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:10.259084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e86b5-b80e-43db-a4c3-76e0cf632a2b service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:10.272575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 987/1999] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:10.388497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831d0e5-1d21-4561-8d02-1cf1e64d2e8f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:10.392878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4831d0e5-1d21-4561-8d02-1cf1e64d2e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:10.425502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4831d0e5-1d21-4561-8d02-1cf1e64d2e8f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:10.448100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:10.448988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:10.449879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:10.450158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] PATH_INFO: `/v3/projects/c382d42027224ce8807082329d6a0b1d` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:10.451666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] RBAC: Authorizing `identity:delete_project(project_id=c382d42027224ce8807082329d6a0b1d)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:10.454193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:10.571724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c382d42027224ce8807082329d6a0b1d', 'request_id': 'req-a670a387-2588-4359-9608-ed4e4c9959df'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:10.660262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c382d42027224ce8807082329d6a0b1d', 'request_id': 'req-a670a387-2588-4359-9608-ed4e4c9959df'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:10.666795 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c382d42027224ce8807082329d6a0b1d', 'request_id': 'req-a670a387-2588-4359-9608-ed4e4c9959df'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:10.680423 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] The token cache is being invalidate because project c382d42027224ce8807082329d6a0b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:10.680423 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:10.680691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:10.682413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1013/2000] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:10 2022] DELETE /identity/v3/projects/c382d42027224ce8807082329d6a0b1d => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:10.687572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e86b5-b80e-43db-a4c3-76e0cf632a2b service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:10.699658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e86b5-b80e-43db-a4c3-76e0cf632a2b service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:10.758008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:10.758885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:10.759173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:10.759448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] PATH_INFO: `/v3/users/e771400f43074aeebfa4ceac7f962ed0` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:10.760681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e771400f43074aeebfa4ceac7f962ed0)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:10.762761 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:10.908403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:10.909927 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:10.925033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670a387-2588-4359-9608-ed4e4c9959df admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:10.930687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:10.931573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:10.931854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:10.932125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] PATH_INFO: `/v3/projects/65c2153f8cc54c4192ce4d2c97681dfe` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:10.933425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c2153f8cc54c4192ce4d2c97681dfe)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:10.935470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:10.941032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e771400f43074aeebfa4ceac7f962ed0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:10.949710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e771400f43074aeebfa4ceac7f962ed0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:10.950419 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e771400f43074aeebfa4ceac7f962ed0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:10.966806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e771400f43074aeebfa4ceac7f962ed0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:10.967489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e771400f43074aeebfa4ceac7f962ed0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.056972 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c2153f8cc54c4192ce4d2c97681dfe', 'request_id': 'req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.066235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 988/2001] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:10 2022] DELETE /identity/v3/users/e771400f43074aeebfa4ceac7f962ed0 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:11.107765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c2153f8cc54c4192ce4d2c97681dfe', 'request_id': 'req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.115602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c2153f8cc54c4192ce4d2c97681dfe', 'request_id': 'req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.123672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:11.128029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] The token cache is being invalidate because project 65c2153f8cc54c4192ce4d2c97681dfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:11.128251 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.128910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.130703 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1014/2002] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:10 2022] DELETE /identity/v3/projects/65c2153f8cc54c4192ce4d2c97681dfe => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:11.134480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:11.135905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:11.191587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:11.209747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c6758-16b5-492f-a0d9-2a3c38551d6a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:11.219114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c5f70-9fe9-4ba4-9a0a-457a4cf11796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:11.219993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c5f70-9fe9-4ba4-9a0a-457a4cf11796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:11.220282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c5f70-9fe9-4ba4-9a0a-457a4cf11796 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:11.220561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c5f70-9fe9-4ba4-9a0a-457a4cf11796 admin admin] PATH_INFO: `/v3/users/06077abaaa2241d3b07ef403506f1d49` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:11.221783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c5f70-9fe9-4ba4-9a0a-457a4cf11796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06077abaaa2241d3b07ef403506f1d49)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:11.223712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c5f70-9fe9-4ba4-9a0a-457a4cf11796 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:11.393834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c67c5f70-9fe9-4ba4-9a0a-457a4cf11796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06077abaaa2241d3b07ef403506f1d49'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.400540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c67c5f70-9fe9-4ba4-9a0a-457a4cf11796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06077abaaa2241d3b07ef403506f1d49'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.401388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c67c5f70-9fe9-4ba4-9a0a-457a4cf11796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06077abaaa2241d3b07ef403506f1d49'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.413820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c67c5f70-9fe9-4ba4-9a0a-457a4cf11796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06077abaaa2241d3b07ef403506f1d49'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.414589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c67c5f70-9fe9-4ba4-9a0a-457a4cf11796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06077abaaa2241d3b07ef403506f1d49'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.496162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:11.502020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:11.520948 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 989/2003] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:11 2022] DELETE /identity/v3/users/06077abaaa2241d3b07ef403506f1d49 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:11.575891 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e6a6d-899e-4cdd-99c8-71a49588d8ee admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:11.644351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:11.645257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:11.645554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:11.645865 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] PATH_INFO: `/v3/projects/01b10169cfd946088945f65ab9e37ad3` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:11.647053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b10169cfd946088945f65ab9e37ad3)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:11.649074 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:11.783407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b10169cfd946088945f65ab9e37ad3', 'request_id': 'req-7477d320-a4e6-415c-926e-17517b9355c3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.877480 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b10169cfd946088945f65ab9e37ad3', 'request_id': 'req-7477d320-a4e6-415c-926e-17517b9355c3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.885819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b10169cfd946088945f65ab9e37ad3', 'request_id': 'req-7477d320-a4e6-415c-926e-17517b9355c3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.898166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] The token cache is being invalidate because project 01b10169cfd946088945f65ab9e37ad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:11.898451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.899274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7477d320-a4e6-415c-926e-17517b9355c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:11.900796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1015/2004] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:11 2022] DELETE /identity/v3/projects/01b10169cfd946088945f65ab9e37ad3 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:11.905756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6432d7d-f132-419f-ab2f-9aa46a686311 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:11.906203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6432d7d-f132-419f-ab2f-9aa46a686311 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:11.906368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6432d7d-f132-419f-ab2f-9aa46a686311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:11.924796 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f6432d7d-f132-419f-ab2f-9aa46a686311 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:11.926032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f6432d7d-f132-419f-ab2f-9aa46a686311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:11.986463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 990/2005] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:11.994394 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b7228-ea49-4dcb-b45c-75043860ef95 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:11.994694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b7228-ea49-4dcb-b45c-75043860ef95 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:11.994975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b7228-ea49-4dcb-b45c-75043860ef95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:11.997622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6432d7d-f132-419f-ab2f-9aa46a686311 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:11.999182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f6432d7d-f132-419f-ab2f-9aa46a686311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:12.022204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6432d7d-f132-419f-ab2f-9aa46a686311 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.022346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-cd8b7228-ea49-4dcb-b45c-75043860ef95 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:12.022346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-cd8b7228-ea49-4dcb-b45c-75043860ef95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:12.022508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.022508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.022508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.022508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] PATH_INFO: `/v3/projects/b59692d38c6544fcbc099e9d3f89e0ed` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.022831 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b59692d38c6544fcbc099e9d3f89e0ed)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.023169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.033006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1016/2006] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:12.038818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b7228-ea49-4dcb-b45c-75043860ef95 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.051022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b7228-ea49-4dcb-b45c-75043860ef95 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.056418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6640c6-f958-41f7-b573-833b3af7d109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.057171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6640c6-f958-41f7-b573-833b3af7d109 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.057562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6640c6-f958-41f7-b573-833b3af7d109 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.057943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6640c6-f958-41f7-b573-833b3af7d109 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.059461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6640c6-f958-41f7-b573-833b3af7d109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:12.059823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6640c6-f958-41f7-b573-833b3af7d109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.061487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6640c6-f958-41f7-b573-833b3af7d109 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.073262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1017/2007] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:57:12.077470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6640c6-f958-41f7-b573-833b3af7d109 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.086731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6640c6-f958-41f7-b573-833b3af7d109 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.091408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc44a68-6d2a-493a-830a-0f4420090c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.092037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc44a68-6d2a-493a-830a-0f4420090c33 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.092326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc44a68-6d2a-493a-830a-0f4420090c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.092604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc44a68-6d2a-493a-830a-0f4420090c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.093326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc44a68-6d2a-493a-830a-0f4420090c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.095196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc44a68-6d2a-493a-830a-0f4420090c33 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.133777 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b59692d38c6544fcbc099e9d3f89e0ed', 'request_id': 'req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:12.198682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1018/2008] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:12 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:12.202837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc44a68-6d2a-493a-830a-0f4420090c33 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.220041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b59692d38c6544fcbc099e9d3f89e0ed', 'request_id': 'req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:12.226497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b59692d38c6544fcbc099e9d3f89e0ed', 'request_id': 'req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:12.241512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] The token cache is being invalidate because project b59692d38c6544fcbc099e9d3f89e0ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:12.241726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:12.242384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-70ae3dbc-3a3f-4949-b070-a4cf1ee5f3b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:12.243983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 991/2009] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:11 2022] DELETE /identity/v3/projects/b59692d38c6544fcbc099e9d3f89e0ed => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:12.249467 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a56487-ec9b-4744-bec0-49d10d1d3400 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.249760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a56487-ec9b-4744-bec0-49d10d1d3400 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.250039 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a56487-ec9b-4744-bec0-49d10d1d3400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.267712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-52a56487-ec9b-4744-bec0-49d10d1d3400 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:12.268936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-52a56487-ec9b-4744-bec0-49d10d1d3400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:12.270175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc44a68-6d2a-493a-830a-0f4420090c33 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.275324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1969e-90bf-4f4e-be8c-20b293c91a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.276139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1969e-90bf-4f4e-be8c-20b293c91a86 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.276427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1969e-90bf-4f4e-be8c-20b293c91a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.280630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf1969e-90bf-4f4e-be8c-20b293c91a86 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.280746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 992/2010] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:12.280839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1969e-90bf-4f4e-be8c-20b293c91a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.280839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf1969e-90bf-4f4e-be8c-20b293c91a86 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.284517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0649bc-32af-4c03-a942-f38a23b889d1 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.284809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0649bc-32af-4c03-a942-f38a23b889d1 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.285091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0649bc-32af-4c03-a942-f38a23b889d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.300466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:12.302771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1a0649bc-32af-4c03-a942-f38a23b889d1 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:12.303955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1a0649bc-32af-4c03-a942-f38a23b889d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:12.311836 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 993/2011] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:12.315869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0649bc-32af-4c03-a942-f38a23b889d1 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.324924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0649bc-32af-4c03-a942-f38a23b889d1 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.329500 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5743c1-07b7-4455-9399-337b4e2c1b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.330124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5743c1-07b7-4455-9399-337b4e2c1b29 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.334203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5743c1-07b7-4455-9399-337b4e2c1b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.334203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5743c1-07b7-4455-9399-337b4e2c1b29 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.334203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5743c1-07b7-4455-9399-337b4e2c1b29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:12.334203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5743c1-07b7-4455-9399-337b4e2c1b29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.334203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5743c1-07b7-4455-9399-337b4e2c1b29 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.344921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 994/2012] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:57:12.349161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5743c1-07b7-4455-9399-337b4e2c1b29 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.359104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1019/2013] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:12 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:12.363103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5743c1-07b7-4455-9399-337b4e2c1b29 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.366417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1969e-90bf-4f4e-be8c-20b293c91a86 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.369009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-bbf1969e-90bf-4f4e-be8c-20b293c91a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:12.370168 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:12.370742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09097d-eabe-4075-b082-670206fc5c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.371945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09097d-eabe-4075-b082-670206fc5c09 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.372230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09097d-eabe-4075-b082-670206fc5c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.372504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09097d-eabe-4075-b082-670206fc5c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.373193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09097d-eabe-4075-b082-670206fc5c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.375113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09097d-eabe-4075-b082-670206fc5c09 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.388781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf1969e-90bf-4f4e-be8c-20b293c91a86 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.395977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f1d54-bca9-4f92-b67c-ec4bafb3e0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.396682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1d54-bca9-4f92-b67c-ec4bafb3e0db admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.396962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1d54-bca9-4f92-b67c-ec4bafb3e0db admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.397292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1d54-bca9-4f92-b67c-ec4bafb3e0db admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.398002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f1d54-bca9-4f92-b67c-ec4bafb3e0db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.399657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f1d54-bca9-4f92-b67c-ec4bafb3e0db admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.411391 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1020/2014] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:12.416959 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f1d54-bca9-4f92-b67c-ec4bafb3e0db admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.426526 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f1d54-bca9-4f92-b67c-ec4bafb3e0db admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.431135 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aca1f7-ffc7-423d-b672-52c5e59e3d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.431765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aca1f7-ffc7-423d-b672-52c5e59e3d70 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.432048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aca1f7-ffc7-423d-b672-52c5e59e3d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.432325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aca1f7-ffc7-423d-b672-52c5e59e3d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.433028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aca1f7-ffc7-423d-b672-52c5e59e3d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.435168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aca1f7-ffc7-423d-b672-52c5e59e3d70 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.445233 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1021/2015] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:12.449603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aca1f7-ffc7-423d-b672-52c5e59e3d70 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.458824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aca1f7-ffc7-423d-b672-52c5e59e3d70 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.463338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e4723-cbe7-46e8-8ec7-7a03dd352c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.464198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e4723-cbe7-46e8-8ec7-7a03dd352c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.464481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e4723-cbe7-46e8-8ec7-7a03dd352c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.464936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e4723-cbe7-46e8-8ec7-7a03dd352c92 admin admin] PATH_INFO: `/v3/projects/bfca06cb219443c39ab409c62d95c03e/users/193019a524254a159aec8f7b9a47ab91/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.479540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e4723-cbe7-46e8-8ec7-7a03dd352c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfca06cb219443c39ab409c62d95c03e, user_id=193019a524254a159aec8f7b9a47ab91, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.483460 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e4723-cbe7-46e8-8ec7-7a03dd352c92 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.506402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 995/2016] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:12 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:12.506402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09097d-eabe-4075-b082-670206fc5c09 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.557841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1022/2017] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:12 2022] PUT /identity/v3/projects/bfca06cb219443c39ab409c62d95c03e/users/193019a524254a159aec8f7b9a47ab91/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:12.567989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398caa01-7fe8-47b1-aa41-a6fee8122d93 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.568281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398caa01-7fe8-47b1-aa41-a6fee8122d93 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.568564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398caa01-7fe8-47b1-aa41-a6fee8122d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.571331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09097d-eabe-4075-b082-670206fc5c09 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.596301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-398caa01-7fe8-47b1-aa41-a6fee8122d93 None None] MFA Rules not processed for user `193019a524254a159aec8f7b9a47ab91`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:12.597610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-398caa01-7fe8-47b1-aa41-a6fee8122d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:12.629988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611227e2-26a5-46d5-b7c2-459582bb09fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.630681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611227e2-26a5-46d5-b7c2-459582bb09fd admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.630969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611227e2-26a5-46d5-b7c2-459582bb09fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.631249 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611227e2-26a5-46d5-b7c2-459582bb09fd admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.631936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611227e2-26a5-46d5-b7c2-459582bb09fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.633900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611227e2-26a5-46d5-b7c2-459582bb09fd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.663566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1023/2018] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:12.667972 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398caa01-7fe8-47b1-aa41-a6fee8122d93 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.679455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398caa01-7fe8-47b1-aa41-a6fee8122d93 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.685569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713855-4266-4363-a615-faaa7036f687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.686624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713855-4266-4363-a615-faaa7036f687 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.687119 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713855-4266-4363-a615-faaa7036f687 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.687415 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09713855-4266-4363-a615-faaa7036f687 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.688087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713855-4266-4363-a615-faaa7036f687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.690010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09713855-4266-4363-a615-faaa7036f687 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.712368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 996/2019] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:12 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:12.716385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611227e2-26a5-46d5-b7c2-459582bb09fd admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.726266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611227e2-26a5-46d5-b7c2-459582bb09fd admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.731227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed3f6a-f3b0-4fce-829c-3e823e098601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.731885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed3f6a-f3b0-4fce-829c-3e823e098601 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.732168 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed3f6a-f3b0-4fce-829c-3e823e098601 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.732446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed3f6a-f3b0-4fce-829c-3e823e098601 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.734707 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed3f6a-f3b0-4fce-829c-3e823e098601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.736310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed3f6a-f3b0-4fce-829c-3e823e098601 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.746890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 997/2020] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:12.752121 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed3f6a-f3b0-4fce-829c-3e823e098601 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.761489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed3f6a-f3b0-4fce-829c-3e823e098601 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.766565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278650cd-1512-4b0d-8d75-039fb524656a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.767217 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278650cd-1512-4b0d-8d75-039fb524656a admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.767485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278650cd-1512-4b0d-8d75-039fb524656a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.767765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278650cd-1512-4b0d-8d75-039fb524656a admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.768456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278650cd-1512-4b0d-8d75-039fb524656a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.770053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278650cd-1512-4b0d-8d75-039fb524656a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.779073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1024/2021] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:12 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:12.780298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713855-4266-4363-a615-faaa7036f687 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.780769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 998/2022] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:12.784563 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278650cd-1512-4b0d-8d75-039fb524656a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.794618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278650cd-1512-4b0d-8d75-039fb524656a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.847457 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09713855-4266-4363-a615-faaa7036f687 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.852611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea2f66-153b-46d5-8986-e7cfd7a141ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.853182 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ebce-417d-4774-b053-9bcf74bdfcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.853570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea2f66-153b-46d5-8986-e7cfd7a141ed admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.853861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea2f66-153b-46d5-8986-e7cfd7a141ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.854129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea2f66-153b-46d5-8986-e7cfd7a141ed admin admin] PATH_INFO: `/v3/projects/229f5509f8bd4783a4bf6d706dd018ee/users/96a6161a7cc34e41bf8618a8e5a89857/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.854825 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ebce-417d-4774-b053-9bcf74bdfcb5 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.856021 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ebce-417d-4774-b053-9bcf74bdfcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.856316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098ebce-417d-4774-b053-9bcf74bdfcb5 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.857158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098ebce-417d-4774-b053-9bcf74bdfcb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.859277 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098ebce-417d-4774-b053-9bcf74bdfcb5 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.875724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea2f66-153b-46d5-8986-e7cfd7a141ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=229f5509f8bd4783a4bf6d706dd018ee, user_id=96a6161a7cc34e41bf8618a8e5a89857, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.878439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea2f66-153b-46d5-8986-e7cfd7a141ed admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.948993 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1025/2023] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:12 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:12.950093 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ebce-417d-4774-b053-9bcf74bdfcb5 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.959770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098ebce-417d-4774-b053-9bcf74bdfcb5 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:12.966835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c5f23-27ed-42b5-a5e4-b8dc64464ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:12.967517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c5f23-27ed-42b5-a5e4-b8dc64464ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.967813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c5f23-27ed-42b5-a5e4-b8dc64464ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.968097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c5f23-27ed-42b5-a5e4-b8dc64464ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.968853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c5f23-27ed-42b5-a5e4-b8dc64464ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:12.969917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 999/2024] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:12 2022] PUT /identity/v3/projects/229f5509f8bd4783a4bf6d706dd018ee/users/96a6161a7cc34e41bf8618a8e5a89857/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:12.970752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10c5f23-27ed-42b5-a5e4-b8dc64464ba8 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:12.984311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1026/2025] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:12.989473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c5f23-27ed-42b5-a5e4-b8dc64464ba8 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:12.989583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde6136-1db9-4454-b4f5-618d7e9c84f5 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:12.989583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde6136-1db9-4454-b4f5-618d7e9c84f5 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:12.989583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde6136-1db9-4454-b4f5-618d7e9c84f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:12.996570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c5f23-27ed-42b5-a5e4-b8dc64464ba8 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:13.022679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-cbde6136-1db9-4454-b4f5-618d7e9c84f5 None None] MFA Rules not processed for user `96a6161a7cc34e41bf8618a8e5a89857`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:13.022679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-cbde6136-1db9-4454-b4f5-618d7e9c84f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:13.044732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:13.057464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a595c-f859-48e2-82f2-6aea0fc82713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:13.059002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a595c-f859-48e2-82f2-6aea0fc82713 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:13.059330 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a595c-f859-48e2-82f2-6aea0fc82713 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:13.059614 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a595c-f859-48e2-82f2-6aea0fc82713 admin admin] PATH_INFO: `/v3/projects/f6ea9bf8afde46029854c1d974eae1ba/users/c3a886e55f484b79a165aed13788223a/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:13.073808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a595c-f859-48e2-82f2-6aea0fc82713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ea9bf8afde46029854c1d974eae1ba, user_id=c3a886e55f484b79a165aed13788223a, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:13.076489 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2a595c-f859-48e2-82f2-6aea0fc82713 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:13.080109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1000/2026] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:13.091137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde6136-1db9-4454-b4f5-618d7e9c84f5 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:13.092467 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-cbde6136-1db9-4454-b4f5-618d7e9c84f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:13.107706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:13.148139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde6136-1db9-4454-b4f5-618d7e9c84f5 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:13.151820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fba79-cd63-4346-86b7-5f77c53a995d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17b7a7128d64b2ba548add02d3ec264', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:13.157365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fba79-cd63-4346-86b7-5f77c53a995d service glance] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:13.157365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fba79-cd63-4346-86b7-5f77c53a995d service glance] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:13.157365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fba79-cd63-4346-86b7-5f77c53a995d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:13.163440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1027/2027] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:12 2022] PUT /identity/v3/projects/f6ea9bf8afde46029854c1d974eae1ba/users/c3a886e55f484b79a165aed13788223a/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:13.164404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a595c-f859-48e2-82f2-6aea0fc82713 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:13.165074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fba79-cd63-4346-86b7-5f77c53a995d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:13.166562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fba79-cd63-4346-86b7-5f77c53a995d service glance] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:13.175801 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1001/2028] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:57:13.227390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a595c-f859-48e2-82f2-6aea0fc82713 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:13.232664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e5dbc-6d11-4b26-b904-bdd55f7f9007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:13.234421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e5dbc-6d11-4b26-b904-bdd55f7f9007 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:13.234421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e5dbc-6d11-4b26-b904-bdd55f7f9007 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:13.234421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e5dbc-6d11-4b26-b904-bdd55f7f9007 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:13.234771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e5dbc-6d11-4b26-b904-bdd55f7f9007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:13.236375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e5dbc-6d11-4b26-b904-bdd55f7f9007 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:13.250665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1028/2029] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:13.251798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fba79-cd63-4346-86b7-5f77c53a995d service glance] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:13.263258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fba79-cd63-4346-86b7-5f77c53a995d service glance] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:13.267972 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6114fa-50ea-411a-8b03-1218087267a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:13.268840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6114fa-50ea-411a-8b03-1218087267a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:13.269119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6114fa-50ea-411a-8b03-1218087267a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:13.269425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6114fa-50ea-411a-8b03-1218087267a8 admin admin] PATH_INFO: `/v3/projects/f6ea9bf8afde46029854c1d974eae1ba/users/c3a886e55f484b79a165aed13788223a/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:13.271368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6114fa-50ea-411a-8b03-1218087267a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ea9bf8afde46029854c1d974eae1ba, user_id=c3a886e55f484b79a165aed13788223a, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:13.274278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6114fa-50ea-411a-8b03-1218087267a8 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:13.286306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1002/2030] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:13 2022] PUT /identity/v3/projects/f6ea9bf8afde46029854c1d974eae1ba/users/c3a886e55f484b79a165aed13788223a/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:13.303013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf15d4d-9919-4999-95c7-baeb7a5d0248 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:13.303519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf15d4d-9919-4999-95c7-baeb7a5d0248 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:13.303807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf15d4d-9919-4999-95c7-baeb7a5d0248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:13.314225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:13.334823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-abf15d4d-9919-4999-95c7-baeb7a5d0248 None None] MFA Rules not processed for user `c3a886e55f484b79a165aed13788223a`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:13.336052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-abf15d4d-9919-4999-95c7-baeb7a5d0248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:13.408363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1029/2031] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:13.420524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6114fa-50ea-411a-8b03-1218087267a8 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:13.422395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ef6114fa-50ea-411a-8b03-1218087267a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:13.449107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:13.489331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6114fa-50ea-411a-8b03-1218087267a8 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:13.492905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c16a47-1cf4-4fda-b190-ce81794d68bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:13.493585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c16a47-1cf4-4fda-b190-ce81794d68bf service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:13.493868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c16a47-1cf4-4fda-b190-ce81794d68bf service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:13.494143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c16a47-1cf4-4fda-b190-ce81794d68bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:13.504499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c16a47-1cf4-4fda-b190-ce81794d68bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:13.506184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c16a47-1cf4-4fda-b190-ce81794d68bf service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:13.517394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1003/2032] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:13.547498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf15d4d-9919-4999-95c7-baeb7a5d0248 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:13.614548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf15d4d-9919-4999-95c7-baeb7a5d0248 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:13.619124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:13.619987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:13.620275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:13.620550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] PATH_INFO: `/v3/users/193019a524254a159aec8f7b9a47ab91` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:13.621778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] RBAC: Authorizing `identity:delete_user(user_id=193019a524254a159aec8f7b9a47ab91)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:13.623757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:13.829101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '193019a524254a159aec8f7b9a47ab91'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:13.829835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '193019a524254a159aec8f7b9a47ab91'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:13.836724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '193019a524254a159aec8f7b9a47ab91'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:13.837378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '193019a524254a159aec8f7b9a47ab91'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:13.849680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '193019a524254a159aec8f7b9a47ab91'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:13.940402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1030/2033] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:13 2022] DELETE /identity/v3/users/193019a524254a159aec8f7b9a47ab91 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:13.945065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5995-b1e6-42d3-a283-6b4bd0485042 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:13.945383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5995-b1e6-42d3-a283-6b4bd0485042 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:13.945660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e5995-b1e6-42d3-a283-6b4bd0485042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:13.963451 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ae5e5995-b1e6-42d3-a283-6b4bd0485042 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:13.964677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ae5e5995-b1e6-42d3-a283-6b4bd0485042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:14.082296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1004/2034] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:14.089754 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:14.091285 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:14.110591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:14.174111 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9bb70-c31c-4063-8ad0-b27f51da431d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:14.177955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01315ac-861b-4ba8-9f2b-dd9ac6a03f5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:14.178614 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01315ac-861b-4ba8-9f2b-dd9ac6a03f5c service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:14.178912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01315ac-861b-4ba8-9f2b-dd9ac6a03f5c service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:14.179181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01315ac-861b-4ba8-9f2b-dd9ac6a03f5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:14.189716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01315ac-861b-4ba8-9f2b-dd9ac6a03f5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:14.190896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01315ac-861b-4ba8-9f2b-dd9ac6a03f5c service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:14.202864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1031/2035] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:14.621145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e5995-b1e6-42d3-a283-6b4bd0485042 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:14.621145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ae5e5995-b1e6-42d3-a283-6b4bd0485042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:14.642904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e5995-b1e6-42d3-a283-6b4bd0485042 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:14.648851 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:14.649890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:14.650285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:14.655246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] PATH_INFO: `/v3/projects/bfca06cb219443c39ab409c62d95c03e` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:14.655246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfca06cb219443c39ab409c62d95c03e)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:14.655246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:14.769708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfca06cb219443c39ab409c62d95c03e', 'request_id': 'req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:14.849553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfca06cb219443c39ab409c62d95c03e', 'request_id': 'req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:14.857103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfca06cb219443c39ab409c62d95c03e', 'request_id': 'req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:14.876461 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] The token cache is being invalidate because project bfca06cb219443c39ab409c62d95c03e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:14.876867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:14.877628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:14.879172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1005/2036] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:14 2022] DELETE /identity/v3/projects/bfca06cb219443c39ab409c62d95c03e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:14.883179 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01315ac-861b-4ba8-9f2b-dd9ac6a03f5c service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:14.895780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01315ac-861b-4ba8-9f2b-dd9ac6a03f5c service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:14.962121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbce13-d7aa-447a-8179-1895547cde78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:14.962996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbce13-d7aa-447a-8179-1895547cde78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:14.963280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbce13-d7aa-447a-8179-1895547cde78 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:14.963556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbce13-d7aa-447a-8179-1895547cde78 admin admin] PATH_INFO: `/v3/users/c3a886e55f484b79a165aed13788223a` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:14.965313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbce13-d7aa-447a-8179-1895547cde78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a886e55f484b79a165aed13788223a)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:14.967467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbce13-d7aa-447a-8179-1895547cde78 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:15.159531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a8cbce13-d7aa-447a-8179-1895547cde78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a886e55f484b79a165aed13788223a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:15.160152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a8cbce13-d7aa-447a-8179-1895547cde78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a886e55f484b79a165aed13788223a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:15.166984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a8cbce13-d7aa-447a-8179-1895547cde78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a886e55f484b79a165aed13788223a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:15.167638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a8cbce13-d7aa-447a-8179-1895547cde78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a886e55f484b79a165aed13788223a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:15.179403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a8cbce13-d7aa-447a-8179-1895547cde78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a886e55f484b79a165aed13788223a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:15.237740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1032/2037] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:14 2022] DELETE /identity/v3/users/c3a886e55f484b79a165aed13788223a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:15.741544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:15.743004 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:15.756473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe8e35-9840-45f8-8b69-ac37bea8acd6 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:15.798518 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:15.814240 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:15.815102 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:15.815388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:15.815665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] PATH_INFO: `/v3/projects/f6ea9bf8afde46029854c1d974eae1ba` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:15.816828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ea9bf8afde46029854c1d974eae1ba)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:15.818776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:15.918409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ea9bf8afde46029854c1d974eae1ba', 'request_id': 'req-79f08010-4961-4fd6-86bd-d4f8a975ac38'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:15.968722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ea9bf8afde46029854c1d974eae1ba', 'request_id': 'req-79f08010-4961-4fd6-86bd-d4f8a975ac38'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:15.975688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ea9bf8afde46029854c1d974eae1ba', 'request_id': 'req-79f08010-4961-4fd6-86bd-d4f8a975ac38'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:16.001429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] The token cache is being invalidate because project f6ea9bf8afde46029854c1d974eae1ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:16.001973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:16.002665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79f08010-4961-4fd6-86bd-d4f8a975ac38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:16.004784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1006/2038] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:15 2022] DELETE /identity/v3/projects/f6ea9bf8afde46029854c1d974eae1ba => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:16.009909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88751a4-010b-4512-b28c-5e6f8900646f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.010370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88751a4-010b-4512-b28c-5e6f8900646f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.010852 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88751a4-010b-4512-b28c-5e6f8900646f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.031185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-e88751a4-010b-4512-b28c-5e6f8900646f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:16.032528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e88751a4-010b-4512-b28c-5e6f8900646f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:16.060252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:16.094866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1033/2039] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:16.103666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce749d66-a28a-4d21-b94d-1796323a2d45 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.103982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce749d66-a28a-4d21-b94d-1796323a2d45 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.104280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce749d66-a28a-4d21-b94d-1796323a2d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.123699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ce749d66-a28a-4d21-b94d-1796323a2d45 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:16.125025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ce749d66-a28a-4d21-b94d-1796323a2d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:16.133625 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1007/2040] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:16.138103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88751a4-010b-4512-b28c-5e6f8900646f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:16.148535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88751a4-010b-4512-b28c-5e6f8900646f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:16.153599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4063222-be7a-431d-8cd4-33f330028edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:16.154245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4063222-be7a-431d-8cd4-33f330028edd admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.154554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4063222-be7a-431d-8cd4-33f330028edd admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.154829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4063222-be7a-431d-8cd4-33f330028edd admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.155557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4063222-be7a-431d-8cd4-33f330028edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:16.155865 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4063222-be7a-431d-8cd4-33f330028edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:16.157486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4063222-be7a-431d-8cd4-33f330028edd admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:16.169576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1034/2041] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:16.173800 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce749d66-a28a-4d21-b94d-1796323a2d45 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:16.183056 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce749d66-a28a-4d21-b94d-1796323a2d45 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:16.187832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871eae4-b5c5-49b4-b08a-72f96ad5dda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:16.188567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871eae4-b5c5-49b4-b08a-72f96ad5dda8 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.188850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871eae4-b5c5-49b4-b08a-72f96ad5dda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.189123 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871eae4-b5c5-49b4-b08a-72f96ad5dda8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.189844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871eae4-b5c5-49b4-b08a-72f96ad5dda8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:16.191748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871eae4-b5c5-49b4-b08a-72f96ad5dda8 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:16.278400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1008/2042] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:16 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:16.282429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4063222-be7a-431d-8cd4-33f330028edd admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:16.292253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4063222-be7a-431d-8cd4-33f330028edd admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:16.348233 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a873f48-5ca2-453b-a499-7f96eb7a3c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:16.348904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a873f48-5ca2-453b-a499-7f96eb7a3c6b admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.349192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a873f48-5ca2-453b-a499-7f96eb7a3c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.349505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a873f48-5ca2-453b-a499-7f96eb7a3c6b admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.350188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a873f48-5ca2-453b-a499-7f96eb7a3c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:16.352101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a873f48-5ca2-453b-a499-7f96eb7a3c6b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:16.431832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1035/2043] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:16 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:16.435666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871eae4-b5c5-49b4-b08a-72f96ad5dda8 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:16.445976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871eae4-b5c5-49b4-b08a-72f96ad5dda8 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:16.453703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d2fdf-b770-49b9-978e-918dd9f5cb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:16.454380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d2fdf-b770-49b9-978e-918dd9f5cb54 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.454665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d2fdf-b770-49b9-978e-918dd9f5cb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.454969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103d2fdf-b770-49b9-978e-918dd9f5cb54 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.456272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d2fdf-b770-49b9-978e-918dd9f5cb54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:16.458255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103d2fdf-b770-49b9-978e-918dd9f5cb54 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:16.475109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1009/2044] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:16.481007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a873f48-5ca2-453b-a499-7f96eb7a3c6b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:16.490980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a873f48-5ca2-453b-a499-7f96eb7a3c6b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:16.496014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccd5a4-bc41-405d-8b95-b95c3d8d279f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:16.496674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccd5a4-bc41-405d-8b95-b95c3d8d279f admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.496964 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccd5a4-bc41-405d-8b95-b95c3d8d279f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.497487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccd5a4-bc41-405d-8b95-b95c3d8d279f admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.498005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccd5a4-bc41-405d-8b95-b95c3d8d279f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:16.499593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ccd5a4-bc41-405d-8b95-b95c3d8d279f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:16.511132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1036/2045] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:16.516576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d2fdf-b770-49b9-978e-918dd9f5cb54 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:16.527539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103d2fdf-b770-49b9-978e-918dd9f5cb54 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:16.532126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314d953-7fb3-4763-b9c7-b96069aae0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:16.533131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314d953-7fb3-4763-b9c7-b96069aae0ad admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.533444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314d953-7fb3-4763-b9c7-b96069aae0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.533722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314d953-7fb3-4763-b9c7-b96069aae0ad admin admin] PATH_INFO: `/v3/projects/4680a5d0b1024125b2ade61d7fc941b9/users/03f33b591ab8443db1c8ed11975ecb94/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.549795 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314d953-7fb3-4763-b9c7-b96069aae0ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=4680a5d0b1024125b2ade61d7fc941b9, user_id=03f33b591ab8443db1c8ed11975ecb94, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:16.552490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314d953-7fb3-4763-b9c7-b96069aae0ad admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:16.644171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1010/2046] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:16 2022] PUT /identity/v3/projects/4680a5d0b1024125b2ade61d7fc941b9/users/03f33b591ab8443db1c8ed11975ecb94/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:16.655091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f5a54-3a39-4450-be30-fae4a6a3c7a9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.655384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f5a54-3a39-4450-be30-fae4a6a3c7a9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.655661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f5a54-3a39-4450-be30-fae4a6a3c7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.685673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-5e5f5a54-3a39-4450-be30-fae4a6a3c7a9 None None] MFA Rules not processed for user `03f33b591ab8443db1c8ed11975ecb94`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:16.686842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5e5f5a54-3a39-4450-be30-fae4a6a3c7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:16.763143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1037/2047] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:16.767683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314d953-7fb3-4763-b9c7-b96069aae0ad admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:16.830679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314d953-7fb3-4763-b9c7-b96069aae0ad admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:16.835385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f45a9b-bb37-406f-ae9b-0a6fb2adfdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:16.836048 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45a9b-bb37-406f-ae9b-0a6fb2adfdbd admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:16.836334 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45a9b-bb37-406f-ae9b-0a6fb2adfdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:16.836613 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f45a9b-bb37-406f-ae9b-0a6fb2adfdbd admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:16.837340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f45a9b-bb37-406f-ae9b-0a6fb2adfdbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:16.839247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f45a9b-bb37-406f-ae9b-0a6fb2adfdbd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:16.852515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:16.946949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1011/2048] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:16 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:16.951002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f5a54-3a39-4450-be30-fae4a6a3c7a9 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:16.961711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f5a54-3a39-4450-be30-fae4a6a3c7a9 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:16.986967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:17.018546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e1bd3-8f85-49aa-ba47-f849939fe012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:17.019223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e1bd3-8f85-49aa-ba47-f849939fe012 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.019503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e1bd3-8f85-49aa-ba47-f849939fe012 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.019777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e1bd3-8f85-49aa-ba47-f849939fe012 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.020455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e1bd3-8f85-49aa-ba47-f849939fe012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:17.022428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e1bd3-8f85-49aa-ba47-f849939fe012 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:17.128666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1038/2049] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:16 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:17.132074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f45a9b-bb37-406f-ae9b-0a6fb2adfdbd admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:17.144368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f45a9b-bb37-406f-ae9b-0a6fb2adfdbd admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:17.156097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a7183-9bc5-4c8a-b456-89f47eb38250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:17.156797 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a7183-9bc5-4c8a-b456-89f47eb38250 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.157086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a7183-9bc5-4c8a-b456-89f47eb38250 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.157401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a7183-9bc5-4c8a-b456-89f47eb38250 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.158138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a7183-9bc5-4c8a-b456-89f47eb38250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:17.160410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070a7183-9bc5-4c8a-b456-89f47eb38250 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:17.173175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1012/2050] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:17.178702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e1bd3-8f85-49aa-ba47-f849939fe012 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:17.188807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e1bd3-8f85-49aa-ba47-f849939fe012 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:17.193772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a68b6-8774-4ba8-a303-11032817da92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:17.194568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a68b6-8774-4ba8-a303-11032817da92 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.195635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a68b6-8774-4ba8-a303-11032817da92 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.195635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a68b6-8774-4ba8-a303-11032817da92 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.196507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a68b6-8774-4ba8-a303-11032817da92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:17.198591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a68b6-8774-4ba8-a303-11032817da92 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:17.218067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1039/2051] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:17.221979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a7183-9bc5-4c8a-b456-89f47eb38250 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:17.232916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a7183-9bc5-4c8a-b456-89f47eb38250 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:17.238242 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b28f2-10ba-4031-ac8e-081dc3cb000a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:17.239198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b28f2-10ba-4031-ac8e-081dc3cb000a admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.239567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b28f2-10ba-4031-ac8e-081dc3cb000a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.239953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b28f2-10ba-4031-ac8e-081dc3cb000a admin admin] PATH_INFO: `/v3/projects/2a57fbb665b74610881ccb35f6095ce6/users/79ccdbb37323432499444ffab6cd2e09/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.258662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b28f2-10ba-4031-ac8e-081dc3cb000a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a57fbb665b74610881ccb35f6095ce6, user_id=79ccdbb37323432499444ffab6cd2e09, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:17.261854 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b28f2-10ba-4031-ac8e-081dc3cb000a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:17.334337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1013/2052] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:17 2022] PUT /identity/v3/projects/2a57fbb665b74610881ccb35f6095ce6/users/79ccdbb37323432499444ffab6cd2e09/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:17.345116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d6790-433f-46cf-9b2f-d5a84574a6b9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.345549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d6790-433f-46cf-9b2f-d5a84574a6b9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.345926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d6790-433f-46cf-9b2f-d5a84574a6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.372819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-af2d6790-433f-46cf-9b2f-d5a84574a6b9 None None] MFA Rules not processed for user `79ccdbb37323432499444ffab6cd2e09`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:17.374074 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-af2d6790-433f-46cf-9b2f-d5a84574a6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:17.437169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1040/2053] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:17.441803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b28f2-10ba-4031-ac8e-081dc3cb000a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:17.506415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b28f2-10ba-4031-ac8e-081dc3cb000a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:17.511092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60a71b-f3b8-4525-996f-decc3d582821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:17.511766 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60a71b-f3b8-4525-996f-decc3d582821 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.512053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60a71b-f3b8-4525-996f-decc3d582821 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.512331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60a71b-f3b8-4525-996f-decc3d582821 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.513089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60a71b-f3b8-4525-996f-decc3d582821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:17.515057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60a71b-f3b8-4525-996f-decc3d582821 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:17.588030 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1014/2054] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:17 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:17.601761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d6790-433f-46cf-9b2f-d5a84574a6b9 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:17.612925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d6790-433f-46cf-9b2f-d5a84574a6b9 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:17.668427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d440eb-2491-4c33-b37d-9576c8697cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:17.669089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d440eb-2491-4c33-b37d-9576c8697cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.669401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d440eb-2491-4c33-b37d-9576c8697cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.669679 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d440eb-2491-4c33-b37d-9576c8697cf1 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.670355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d440eb-2491-4c33-b37d-9576c8697cf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:17.672270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d440eb-2491-4c33-b37d-9576c8697cf1 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:17.746722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1041/2055] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:17 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:17.750542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60a71b-f3b8-4525-996f-decc3d582821 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:17.761004 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60a71b-f3b8-4525-996f-decc3d582821 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:17.765832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297405d-3061-4d87-a13f-5a3517fa4f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:17.766493 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297405d-3061-4d87-a13f-5a3517fa4f23 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.771190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297405d-3061-4d87-a13f-5a3517fa4f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.771190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297405d-3061-4d87-a13f-5a3517fa4f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.771190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297405d-3061-4d87-a13f-5a3517fa4f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:17.771190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6297405d-3061-4d87-a13f-5a3517fa4f23 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:17.784443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1015/2056] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:17.786060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d440eb-2491-4c33-b37d-9576c8697cf1 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:17.796978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d440eb-2491-4c33-b37d-9576c8697cf1 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:17.801834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d598bf5-b93b-4b61-a08e-05aaf234d684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:17.802746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598bf5-b93b-4b61-a08e-05aaf234d684 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.803033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598bf5-b93b-4b61-a08e-05aaf234d684 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.803357 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d598bf5-b93b-4b61-a08e-05aaf234d684 admin admin] PATH_INFO: `/v3/projects/fa7a11a257c242b0bfcfe7cfb1db6157/users/52c51d547f1d4ca5be20d7565dc5a511/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.818104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d598bf5-b93b-4b61-a08e-05aaf234d684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa7a11a257c242b0bfcfe7cfb1db6157, user_id=52c51d547f1d4ca5be20d7565dc5a511, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:17.820736 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d598bf5-b93b-4b61-a08e-05aaf234d684 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:17.919823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:17.925169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1042/2057] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:17 2022] PUT /identity/v3/projects/fa7a11a257c242b0bfcfe7cfb1db6157/users/52c51d547f1d4ca5be20d7565dc5a511/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:17.929230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297405d-3061-4d87-a13f-5a3517fa4f23 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:17.939499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297405d-3061-4d87-a13f-5a3517fa4f23 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:17.996126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f7e67-8f94-42b8-8c21-c0acf1fb7b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:17.996856 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f7e67-8f94-42b8-8c21-c0acf1fb7b93 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:17.997090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f7e67-8f94-42b8-8c21-c0acf1fb7b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:17.997402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f7e67-8f94-42b8-8c21-c0acf1fb7b93 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:17.998123 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f7e67-8f94-42b8-8c21-c0acf1fb7b93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:17.999716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f7e67-8f94-42b8-8c21-c0acf1fb7b93 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:18.009468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1016/2058] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:18.014338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d598bf5-b93b-4b61-a08e-05aaf234d684 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:18.024892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d598bf5-b93b-4b61-a08e-05aaf234d684 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:18.030216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e9d92-a5f4-4259-a277-ec7eac8c7460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:18.031085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e9d92-a5f4-4259-a277-ec7eac8c7460 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:18.031490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e9d92-a5f4-4259-a277-ec7eac8c7460 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:18.031771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e9d92-a5f4-4259-a277-ec7eac8c7460 admin admin] PATH_INFO: `/v3/projects/fa7a11a257c242b0bfcfe7cfb1db6157/users/52c51d547f1d4ca5be20d7565dc5a511/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:18.033778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e9d92-a5f4-4259-a277-ec7eac8c7460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa7a11a257c242b0bfcfe7cfb1db6157, user_id=52c51d547f1d4ca5be20d7565dc5a511, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:18.036487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e9d92-a5f4-4259-a277-ec7eac8c7460 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:18.047800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1043/2059] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:18 2022] PUT /identity/v3/projects/fa7a11a257c242b0bfcfe7cfb1db6157/users/52c51d547f1d4ca5be20d7565dc5a511/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:18.058737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff38264-2dc3-406a-b0f8-4f8fa7b8bc38 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:18.059033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff38264-2dc3-406a-b0f8-4f8fa7b8bc38 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:18.059319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff38264-2dc3-406a-b0f8-4f8fa7b8bc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:18.085798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-4ff38264-2dc3-406a-b0f8-4f8fa7b8bc38 None None] MFA Rules not processed for user `52c51d547f1d4ca5be20d7565dc5a511`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:18.087050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-4ff38264-2dc3-406a-b0f8-4f8fa7b8bc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:18.103260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:18.155507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1017/2060] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:18.166118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e9d92-a5f4-4259-a277-ec7eac8c7460 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:18.168571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3b3e9d92-a5f4-4259-a277-ec7eac8c7460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:18.228247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e9d92-a5f4-4259-a277-ec7eac8c7460 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:18.232507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128de1ea-a029-4887-848b-432100cf38ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:18.235550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128de1ea-a029-4887-848b-432100cf38ae service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:18.235550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128de1ea-a029-4887-848b-432100cf38ae service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:18.235550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128de1ea-a029-4887-848b-432100cf38ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:18.244055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128de1ea-a029-4887-848b-432100cf38ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:18.245607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128de1ea-a029-4887-848b-432100cf38ae service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:18.258797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1044/2061] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:19.371311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff38264-2dc3-406a-b0f8-4f8fa7b8bc38 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:19.372791 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-4ff38264-2dc3-406a-b0f8-4f8fa7b8bc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:19.385974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff38264-2dc3-406a-b0f8-4f8fa7b8bc38 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:19.389837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946543e0-1af2-44b7-852f-df5bfe88bb9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:19.390575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946543e0-1af2-44b7-852f-df5bfe88bb9f service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:19.390963 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946543e0-1af2-44b7-852f-df5bfe88bb9f service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:19.391356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946543e0-1af2-44b7-852f-df5bfe88bb9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:19.453967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946543e0-1af2-44b7-852f-df5bfe88bb9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:19.455549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946543e0-1af2-44b7-852f-df5bfe88bb9f service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:19.465504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1018/2062] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:22.916505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d8677-a5ca-4137-b73b-07bcc5c3381e None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:22.917038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d8677-a5ca-4137-b73b-07bcc5c3381e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:22.917223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d8677-a5ca-4137-b73b-07bcc5c3381e None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:22.919505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1045/2063] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 18:57:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:57:22.952913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c43e3-9335-450e-9387-df7c19ce164a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:22.953420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c43e3-9335-450e-9387-df7c19ce164a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:22.953584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c43e3-9335-450e-9387-df7c19ce164a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:22.973447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1c4c43e3-9335-450e-9387-df7c19ce164a None None] MFA Rules not processed for user `20c0fd976e684356b6143dc33fb7b4af`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:22.974883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1c4c43e3-9335-450e-9387-df7c19ce164a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:23.051546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1019/2064] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 18:57:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:23.063590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d8677-a5ca-4137-b73b-07bcc5c3381e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:23.065575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a28d8677-a5ca-4137-b73b-07bcc5c3381e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:23.101935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d8677-a5ca-4137-b73b-07bcc5c3381e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:23.104596 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c43e3-9335-450e-9387-df7c19ce164a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:23.106906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1c4c43e3-9335-450e-9387-df7c19ce164a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:23.109158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaae565-573b-4299-8070-499bbdba15d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:23.110026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaae565-573b-4299-8070-499bbdba15d2 service placement] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:23.110318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaae565-573b-4299-8070-499bbdba15d2 service placement] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:23.110600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaae565-573b-4299-8070-499bbdba15d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:23.123141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaae565-573b-4299-8070-499bbdba15d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:23.126026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaae565-573b-4299-8070-499bbdba15d2 service placement] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:23.142625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1046/2065] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:23.176660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:23.192057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c43e3-9335-450e-9387-df7c19ce164a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:23.196747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ee144-dc69-49e7-a9cb-bfcd19dd3813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:23.197671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ee144-dc69-49e7-a9cb-bfcd19dd3813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:23.197953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ee144-dc69-49e7-a9cb-bfcd19dd3813 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:23.198228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ee144-dc69-49e7-a9cb-bfcd19dd3813 admin admin] PATH_INFO: `/v3/users/1052dda92d1a4597be43a7acdcd06905` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:23.199425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ee144-dc69-49e7-a9cb-bfcd19dd3813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1052dda92d1a4597be43a7acdcd06905)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:23.201632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ee144-dc69-49e7-a9cb-bfcd19dd3813 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:23.416537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3a7ee144-dc69-49e7-a9cb-bfcd19dd3813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1052dda92d1a4597be43a7acdcd06905'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:23.423167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3a7ee144-dc69-49e7-a9cb-bfcd19dd3813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1052dda92d1a4597be43a7acdcd06905'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:23.423821 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3a7ee144-dc69-49e7-a9cb-bfcd19dd3813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1052dda92d1a4597be43a7acdcd06905'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:23.435453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3a7ee144-dc69-49e7-a9cb-bfcd19dd3813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1052dda92d1a4597be43a7acdcd06905'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:23.436086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3a7ee144-dc69-49e7-a9cb-bfcd19dd3813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1052dda92d1a4597be43a7acdcd06905'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:23.499552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1020/2066] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:23 2022] DELETE /identity/v3/users/1052dda92d1a4597be43a7acdcd06905 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:23.736689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaae565-573b-4299-8070-499bbdba15d2 service placement] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:23.750015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaae565-573b-4299-8070-499bbdba15d2 service placement] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:23.820242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:23.821105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:23.821428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:23.821706 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] PATH_INFO: `/v3/projects/57f75a8c58e340a1bba335f7463c03a1` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:23.822867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f75a8c58e340a1bba335f7463c03a1)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:23.825451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:24.037368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f75a8c58e340a1bba335f7463c03a1', 'request_id': 'req-7516f902-08da-4fde-89c0-df697cbcd168'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:24.094192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f75a8c58e340a1bba335f7463c03a1', 'request_id': 'req-7516f902-08da-4fde-89c0-df697cbcd168'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:24.101015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f75a8c58e340a1bba335f7463c03a1', 'request_id': 'req-7516f902-08da-4fde-89c0-df697cbcd168'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:24.113336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] The token cache is being invalidate because project 57f75a8c58e340a1bba335f7463c03a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:24.113764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:24.114448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7516f902-08da-4fde-89c0-df697cbcd168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:24.115977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1047/2067] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:23 2022] DELETE /identity/v3/projects/57f75a8c58e340a1bba335f7463c03a1 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:24.121769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559076e-655d-4175-aa12-fff4e5dd1513 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.122280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559076e-655d-4175-aa12-fff4e5dd1513 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.122379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559076e-655d-4175-aa12-fff4e5dd1513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.142177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-e559076e-655d-4175-aa12-fff4e5dd1513 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:24.145439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e559076e-655d-4175-aa12-fff4e5dd1513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:24.231229 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1021/2068] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:24.239466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1ee8f-ff86-4c9e-bbf1-50d6b5333159 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.239870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559076e-655d-4175-aa12-fff4e5dd1513 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:24.240306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1ee8f-ff86-4c9e-bbf1-50d6b5333159 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.240856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1ee8f-ff86-4c9e-bbf1-50d6b5333159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.242282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e559076e-655d-4175-aa12-fff4e5dd1513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:24.257615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:24.266054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9ce1ee8f-ff86-4c9e-bbf1-50d6b5333159 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:24.267426 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9ce1ee8f-ff86-4c9e-bbf1-50d6b5333159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:24.270074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559076e-655d-4175-aa12-fff4e5dd1513 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:24.279426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487b387-5ce8-46ca-bffb-1fd35bf7966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:24.280656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487b387-5ce8-46ca-bffb-1fd35bf7966f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.281196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487b387-5ce8-46ca-bffb-1fd35bf7966f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.281734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487b387-5ce8-46ca-bffb-1fd35bf7966f admin admin] PATH_INFO: `/v3/users/c40751576420451c9fba2aebcffaf2f4` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.283308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1048/2069] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:24.283848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1ee8f-ff86-4c9e-bbf1-50d6b5333159 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:24.291766 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487b387-5ce8-46ca-bffb-1fd35bf7966f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c40751576420451c9fba2aebcffaf2f4)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:24.294574 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5487b387-5ce8-46ca-bffb-1fd35bf7966f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:24.295026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1ee8f-ff86-4c9e-bbf1-50d6b5333159 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:24.300476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c06004-5781-4a33-ac74-87aee34e1883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:24.301364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c06004-5781-4a33-ac74-87aee34e1883 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.301655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c06004-5781-4a33-ac74-87aee34e1883 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.301938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c06004-5781-4a33-ac74-87aee34e1883 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.302698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c06004-5781-4a33-ac74-87aee34e1883 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:24.303014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c06004-5781-4a33-ac74-87aee34e1883 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:24.304888 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c06004-5781-4a33-ac74-87aee34e1883 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:24.324332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1049/2070] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:24.329198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c06004-5781-4a33-ac74-87aee34e1883 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:24.340812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c06004-5781-4a33-ac74-87aee34e1883 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:24.350807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b42a37-6cd4-4aeb-b218-1bc93e31474d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:24.351520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b42a37-6cd4-4aeb-b218-1bc93e31474d admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.351811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b42a37-6cd4-4aeb-b218-1bc93e31474d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.352333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b42a37-6cd4-4aeb-b218-1bc93e31474d admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.353048 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b42a37-6cd4-4aeb-b218-1bc93e31474d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:24.358656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b42a37-6cd4-4aeb-b218-1bc93e31474d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:24.456042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5487b387-5ce8-46ca-bffb-1fd35bf7966f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c40751576420451c9fba2aebcffaf2f4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:24.470520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1050/2071] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:24 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:24.471690 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b42a37-6cd4-4aeb-b218-1bc93e31474d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:24.474271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5487b387-5ce8-46ca-bffb-1fd35bf7966f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c40751576420451c9fba2aebcffaf2f4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:24.475257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5487b387-5ce8-46ca-bffb-1fd35bf7966f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c40751576420451c9fba2aebcffaf2f4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:24.484513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:24.496187 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5487b387-5ce8-46ca-bffb-1fd35bf7966f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c40751576420451c9fba2aebcffaf2f4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:24.497318 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5487b387-5ce8-46ca-bffb-1fd35bf7966f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c40751576420451c9fba2aebcffaf2f4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:24.542817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b42a37-6cd4-4aeb-b218-1bc93e31474d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:24.547474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c39e6-fd4b-496d-94cd-04c4cdff3ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:24.548314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c39e6-fd4b-496d-94cd-04c4cdff3ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.548427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c39e6-fd4b-496d-94cd-04c4cdff3ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.548684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c39e6-fd4b-496d-94cd-04c4cdff3ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.549449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c39e6-fd4b-496d-94cd-04c4cdff3ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:24.553693 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c39e6-fd4b-496d-94cd-04c4cdff3ff5 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:24.578296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1022/2072] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:24 2022] DELETE /identity/v3/users/c40751576420451c9fba2aebcffaf2f4 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:24.579408 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3ea28-fd8e-4ea8-a13d-7a2f75f30e54 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.579699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3ea28-fd8e-4ea8-a13d-7a2f75f30e54 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.580120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3ea28-fd8e-4ea8-a13d-7a2f75f30e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.599957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ecc3ea28-fd8e-4ea8-a13d-7a2f75f30e54 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:24.601170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ecc3ea28-fd8e-4ea8-a13d-7a2f75f30e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:24.653104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1051/2073] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:24 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:24.653104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c39e6-fd4b-496d-94cd-04c4cdff3ff5 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:24.654496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-493c39e6-fd4b-496d-94cd-04c4cdff3ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:24.674482 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1023/2074] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:24.675766 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3ea28-fd8e-4ea8-a13d-7a2f75f30e54 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:24.677439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ecc3ea28-fd8e-4ea8-a13d-7a2f75f30e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:24.695928 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3ea28-fd8e-4ea8-a13d-7a2f75f30e54 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:24.699309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc6207-22b6-4b59-a46b-798498876411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:24.699992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc6207-22b6-4b59-a46b-798498876411 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.700287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc6207-22b6-4b59-a46b-798498876411 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.700572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc6207-22b6-4b59-a46b-798498876411 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.701321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc6207-22b6-4b59-a46b-798498876411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:24.702940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc6207-22b6-4b59-a46b-798498876411 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:24.717344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1024/2075] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:24.717344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc6207-22b6-4b59-a46b-798498876411 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:24.717344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-13cc6207-22b6-4b59-a46b-798498876411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:24.723367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c39e6-fd4b-496d-94cd-04c4cdff3ff5 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:24.786368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc6207-22b6-4b59-a46b-798498876411 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:24.786571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d16eb1-4d85-4b39-8701-31d9acbef4ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:24.787073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d16eb1-4d85-4b39-8701-31d9acbef4ad service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.787367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d16eb1-4d85-4b39-8701-31d9acbef4ad service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.787644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d16eb1-4d85-4b39-8701-31d9acbef4ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.789582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-98d16eb1-4d85-4b39-8701-31d9acbef4ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:24.794678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70000a95-6aad-4d13-860b-23869adc83be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:24.794678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70000a95-6aad-4d13-860b-23869adc83be service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.794678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70000a95-6aad-4d13-860b-23869adc83be service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.794678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70000a95-6aad-4d13-860b-23869adc83be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.802680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70000a95-6aad-4d13-860b-23869adc83be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:24.804060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70000a95-6aad-4d13-860b-23869adc83be service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:24.817677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1025/2076] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:24.818388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70000a95-6aad-4d13-860b-23869adc83be service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:24.826504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:24.828442 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70000a95-6aad-4d13-860b-23869adc83be service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:24.833820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d15f2b-de2b-42a9-a651-b08324b31b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:24.834850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15f2b-de2b-42a9-a651-b08324b31b3e admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.835142 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15f2b-de2b-42a9-a651-b08324b31b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.835423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d15f2b-de2b-42a9-a651-b08324b31b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.836155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d15f2b-de2b-42a9-a651-b08324b31b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:24.838130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d15f2b-de2b-42a9-a651-b08324b31b3e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:24.852833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1026/2077] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:24.857716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d15f2b-de2b-42a9-a651-b08324b31b3e admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:24.871196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d16eb1-4d85-4b39-8701-31d9acbef4ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:24.874944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d15f2b-de2b-42a9-a651-b08324b31b3e admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:24.875858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d16eb1-4d85-4b39-8701-31d9acbef4ad service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:24.878319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b39c7-d3d7-4a98-bbd4-76ceef15e9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:24.879643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b39c7-d3d7-4a98-bbd4-76ceef15e9fa admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:24.880069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b39c7-d3d7-4a98-bbd4-76ceef15e9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:24.880424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b39c7-d3d7-4a98-bbd4-76ceef15e9fa admin admin] PATH_INFO: `/v3/projects/d135af04e85245efab72dc1f1f1cd245/users/85c4be4dc6104645a7662ce637444f35/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:24.904824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1052/2078] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:24.910700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b39c7-d3d7-4a98-bbd4-76ceef15e9fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d135af04e85245efab72dc1f1f1cd245, user_id=85c4be4dc6104645a7662ce637444f35, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:24.915561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b39c7-d3d7-4a98-bbd4-76ceef15e9fa admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:25.037540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1027/2079] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:24 2022] PUT /identity/v3/projects/d135af04e85245efab72dc1f1f1cd245/users/85c4be4dc6104645a7662ce637444f35/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:25.048109 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3701156-96fa-47cd-bd54-02e1fb527199 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:25.048620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3701156-96fa-47cd-bd54-02e1fb527199 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:25.048720 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3701156-96fa-47cd-bd54-02e1fb527199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:25.078576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-c3701156-96fa-47cd-bd54-02e1fb527199 None None] MFA Rules not processed for user `85c4be4dc6104645a7662ce637444f35`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:25.080026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c3701156-96fa-47cd-bd54-02e1fb527199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:25.157938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1053/2080] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:25.169405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b39c7-d3d7-4a98-bbd4-76ceef15e9fa admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:25.171214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-917b39c7-d3d7-4a98-bbd4-76ceef15e9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:25.222364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:25.257699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b39c7-d3d7-4a98-bbd4-76ceef15e9fa admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:25.263883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075bb8e4-8ab9-471a-8ed0-a26486ad7c05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:25.264580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bb8e4-8ab9-471a-8ed0-a26486ad7c05 service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:25.264894 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bb8e4-8ab9-471a-8ed0-a26486ad7c05 service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:25.265166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bb8e4-8ab9-471a-8ed0-a26486ad7c05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:25.272900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075bb8e4-8ab9-471a-8ed0-a26486ad7c05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:25.274508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075bb8e4-8ab9-471a-8ed0-a26486ad7c05 service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:25.283167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1028/2081] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:25.297006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3701156-96fa-47cd-bd54-02e1fb527199 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:25.299908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c3701156-96fa-47cd-bd54-02e1fb527199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:25.412725 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3701156-96fa-47cd-bd54-02e1fb527199 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:25.417668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:25.418703 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:25.419070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:25.419426 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] PATH_INFO: `/v3/projects/0fcad64e35874948a0dd78fb42ce9404` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:25.420769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fcad64e35874948a0dd78fb42ce9404)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:25.423076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:25.520623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fcad64e35874948a0dd78fb42ce9404', 'request_id': 'req-d491b15f-abc4-46f9-94a8-538045ce20fb'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:25.585797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fcad64e35874948a0dd78fb42ce9404', 'request_id': 'req-d491b15f-abc4-46f9-94a8-538045ce20fb'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:25.587411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:25.593266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fcad64e35874948a0dd78fb42ce9404', 'request_id': 'req-d491b15f-abc4-46f9-94a8-538045ce20fb'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:25.604560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] The token cache is being invalidate because project 0fcad64e35874948a0dd78fb42ce9404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:25.604866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:25.605792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d491b15f-abc4-46f9-94a8-538045ce20fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:25.607338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1054/2082] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:25 2022] DELETE /identity/v3/projects/0fcad64e35874948a0dd78fb42ce9404 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:25.612678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f17cf-8f9d-41a7-8b13-9460898d066b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:25.612997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f17cf-8f9d-41a7-8b13-9460898d066b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:25.613456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f17cf-8f9d-41a7-8b13-9460898d066b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:25.631408 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-146f17cf-8f9d-41a7-8b13-9460898d066b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:25.632608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-146f17cf-8f9d-41a7-8b13-9460898d066b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:25.694789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1029/2083] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:25.703084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96d893-fec2-43db-a774-a27cff549016 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:25.703380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96d893-fec2-43db-a774-a27cff549016 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:25.703677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd96d893-fec2-43db-a774-a27cff549016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:25.722371 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-bd96d893-fec2-43db-a774-a27cff549016 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:25.723658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-bd96d893-fec2-43db-a774-a27cff549016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:25.732490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1055/2084] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:25.736682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f17cf-8f9d-41a7-8b13-9460898d066b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:25.746491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f17cf-8f9d-41a7-8b13-9460898d066b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:25.751153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0873f19-42ef-46d9-b889-875276dedcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:25.752786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0873f19-42ef-46d9-b889-875276dedcaa admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:25.752786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0873f19-42ef-46d9-b889-875276dedcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:25.752786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0873f19-42ef-46d9-b889-875276dedcaa admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:25.753141 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0873f19-42ef-46d9-b889-875276dedcaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:25.753473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0873f19-42ef-46d9-b889-875276dedcaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:25.755436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0873f19-42ef-46d9-b889-875276dedcaa admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:25.767281 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1030/2085] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:25.771564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96d893-fec2-43db-a774-a27cff549016 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:25.781178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd96d893-fec2-43db-a774-a27cff549016 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:25.785808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c0335-8133-4118-af4c-8b2c34248e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:25.786440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c0335-8133-4118-af4c-8b2c34248e4b admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:25.786719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c0335-8133-4118-af4c-8b2c34248e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:25.786991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c0335-8133-4118-af4c-8b2c34248e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:25.787663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c0335-8133-4118-af4c-8b2c34248e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:25.789631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c0335-8133-4118-af4c-8b2c34248e4b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:25.895450 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1056/2086] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:25 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:25.899511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0873f19-42ef-46d9-b889-875276dedcaa admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:25.910697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0873f19-42ef-46d9-b889-875276dedcaa admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:25.987124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f239f-bcd3-4ac9-8fc7-a3784b04d84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:25.987799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f239f-bcd3-4ac9-8fc7-a3784b04d84b admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:25.988086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f239f-bcd3-4ac9-8fc7-a3784b04d84b admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:25.988362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f239f-bcd3-4ac9-8fc7-a3784b04d84b admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:25.989047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f239f-bcd3-4ac9-8fc7-a3784b04d84b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:25.991176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863f239f-bcd3-4ac9-8fc7-a3784b04d84b admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:26.109704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1031/2087] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:25 2022] POST /identity/v3/users => generated 287 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:26.111120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c0335-8133-4118-af4c-8b2c34248e4b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:26.122135 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c0335-8133-4118-af4c-8b2c34248e4b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:26.126620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7572d8-1338-4638-b483-ad50d40a71a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:26.127299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7572d8-1338-4638-b483-ad50d40a71a6 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:26.127585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7572d8-1338-4638-b483-ad50d40a71a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:26.127859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7572d8-1338-4638-b483-ad50d40a71a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:26.128571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7572d8-1338-4638-b483-ad50d40a71a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:26.133848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7572d8-1338-4638-b483-ad50d40a71a6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:26.140699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1057/2088] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:26.144712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f239f-bcd3-4ac9-8fc7-a3784b04d84b admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:26.154843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863f239f-bcd3-4ac9-8fc7-a3784b04d84b admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:26.159484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f3f6f-c3b4-4b03-9123-f8d86b1233ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:26.160135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f3f6f-c3b4-4b03-9123-f8d86b1233ff admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:26.160419 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f3f6f-c3b4-4b03-9123-f8d86b1233ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:26.160691 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f3f6f-c3b4-4b03-9123-f8d86b1233ff admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:26.161569 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f3f6f-c3b4-4b03-9123-f8d86b1233ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:26.163166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f3f6f-c3b4-4b03-9123-f8d86b1233ff admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:26.173747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1032/2089] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:26.177229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7572d8-1338-4638-b483-ad50d40a71a6 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:26.187732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7572d8-1338-4638-b483-ad50d40a71a6 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:26.192237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5904677-bebb-41b4-9116-ae03ca049dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:26.193102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5904677-bebb-41b4-9116-ae03ca049dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:26.193421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5904677-bebb-41b4-9116-ae03ca049dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:26.193735 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5904677-bebb-41b4-9116-ae03ca049dc8 admin admin] PATH_INFO: `/v3/projects/5db8d64fab0e45ffbba87b1ae3b48703/users/0569765b6f07473fa07a5376b3677da8/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:26.210086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5904677-bebb-41b4-9116-ae03ca049dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db8d64fab0e45ffbba87b1ae3b48703, user_id=0569765b6f07473fa07a5376b3677da8, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:26.212722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5904677-bebb-41b4-9116-ae03ca049dc8 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:26.266366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1058/2090] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:26 2022] PUT /identity/v3/projects/5db8d64fab0e45ffbba87b1ae3b48703/users/0569765b6f07473fa07a5376b3677da8/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:26.271394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb1ee9-3e90-49b9-9f15-4b24cc155c7d None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:26.271745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb1ee9-3e90-49b9-9f15-4b24cc155c7d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:26.272024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb1ee9-3e90-49b9-9f15-4b24cc155c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:26.289465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2dcb1ee9-3e90-49b9-9f15-4b24cc155c7d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:26.290662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2dcb1ee9-3e90-49b9-9f15-4b24cc155c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:26.314581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:26.347974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1033/2091] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:26.358351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5904677-bebb-41b4-9116-ae03ca049dc8 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:26.360125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e5904677-bebb-41b4-9116-ae03ca049dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:26.415984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5904677-bebb-41b4-9116-ae03ca049dc8 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:26.420147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f06175-c830-4dc5-a190-c6e6bcef779f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:26.420789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f06175-c830-4dc5-a190-c6e6bcef779f service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:26.421077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f06175-c830-4dc5-a190-c6e6bcef779f service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:26.421390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f06175-c830-4dc5-a190-c6e6bcef779f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:26.430334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f06175-c830-4dc5-a190-c6e6bcef779f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:26.431792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f06175-c830-4dc5-a190-c6e6bcef779f service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:26.441765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1059/2092] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:32.155496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb1ee9-3e90-49b9-9f15-4b24cc155c7d None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:32.158767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2dcb1ee9-3e90-49b9-9f15-4b24cc155c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:32.178998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb1ee9-3e90-49b9-9f15-4b24cc155c7d None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:32.184496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f8090-a5e1-4511-abe4-e06b7ba5ac8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:32.185877 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f8090-a5e1-4511-abe4-e06b7ba5ac8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:32.186445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f8090-a5e1-4511-abe4-e06b7ba5ac8e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:32.186921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f8090-a5e1-4511-abe4-e06b7ba5ac8e admin admin] PATH_INFO: `/v3/users/03f33b591ab8443db1c8ed11975ecb94` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:32.188324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f8090-a5e1-4511-abe4-e06b7ba5ac8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f33b591ab8443db1c8ed11975ecb94)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:32.190644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f8090-a5e1-4511-abe4-e06b7ba5ac8e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:32.254777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497c89e-945c-4161-8a2c-613a945aef06 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:32.255943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497c89e-945c-4161-8a2c-613a945aef06 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:32.256536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497c89e-945c-4161-8a2c-613a945aef06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:32.295095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4497c89e-945c-4161-8a2c-613a945aef06 None None] MFA Rules not processed for user `0569765b6f07473fa07a5376b3677da8`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:32.296421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4497c89e-945c-4161-8a2c-613a945aef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:32.380527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1060/2093] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:32.385253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497c89e-945c-4161-8a2c-613a945aef06 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:32.396409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497c89e-945c-4161-8a2c-613a945aef06 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:32.401867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91aa3a-421b-49f9-84a7-5522ed73c191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:32.402549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91aa3a-421b-49f9-84a7-5522ed73c191 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:32.402837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91aa3a-421b-49f9-84a7-5522ed73c191 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:32.403115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91aa3a-421b-49f9-84a7-5522ed73c191 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:32.403843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91aa3a-421b-49f9-84a7-5522ed73c191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:32.405834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f91aa3a-421b-49f9-84a7-5522ed73c191 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:32.406520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-076f8090-a5e1-4511-abe4-e06b7ba5ac8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f33b591ab8443db1c8ed11975ecb94'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:32.414348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-076f8090-a5e1-4511-abe4-e06b7ba5ac8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f33b591ab8443db1c8ed11975ecb94'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:32.415227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-076f8090-a5e1-4511-abe4-e06b7ba5ac8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f33b591ab8443db1c8ed11975ecb94'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:32.427604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:32.431759 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-076f8090-a5e1-4511-abe4-e06b7ba5ac8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f33b591ab8443db1c8ed11975ecb94'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:32.434727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-076f8090-a5e1-4511-abe4-e06b7ba5ac8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f33b591ab8443db1c8ed11975ecb94'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:32.524638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1061/2094] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:32 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:32.528499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91aa3a-421b-49f9-84a7-5522ed73c191 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:32.530259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-2f91aa3a-421b-49f9-84a7-5522ed73c191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:32.553878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1034/2095] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:32 2022] DELETE /identity/v3/users/03f33b591ab8443db1c8ed11975ecb94 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:32.561856 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b6833-a938-4fdf-a063-84a7933eeb02 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:32.562150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b6833-a938-4fdf-a063-84a7933eeb02 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:32.562426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b6833-a938-4fdf-a063-84a7933eeb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:32.582110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f35b6833-a938-4fdf-a063-84a7933eeb02 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:32.583331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f35b6833-a938-4fdf-a063-84a7933eeb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:32.602229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91aa3a-421b-49f9-84a7-5522ed73c191 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:32.645013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1035/2096] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:32.655525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b6833-a938-4fdf-a063-84a7933eeb02 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:32.657297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f35b6833-a938-4fdf-a063-84a7933eeb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:32.670927 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc38fe-e472-495e-9fe4-adfdf914ff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:32.671697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc38fe-e472-495e-9fe4-adfdf914ff6a admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:32.671966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc38fe-e472-495e-9fe4-adfdf914ff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:32.672247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc38fe-e472-495e-9fe4-adfdf914ff6a admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:32.672949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc38fe-e472-495e-9fe4-adfdf914ff6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:32.675128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc38fe-e472-495e-9fe4-adfdf914ff6a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:32.707730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:32.719842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b6833-a938-4fdf-a063-84a7933eeb02 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:32.723690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfe0dc-ff3e-42f0-ade2-4762e4d1daee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:32.724362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfe0dc-ff3e-42f0-ade2-4762e4d1daee service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:32.724700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfe0dc-ff3e-42f0-ade2-4762e4d1daee service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:32.724963 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfe0dc-ff3e-42f0-ade2-4762e4d1daee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:32.737675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfe0dc-ff3e-42f0-ade2-4762e4d1daee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:32.739092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfe0dc-ff3e-42f0-ade2-4762e4d1daee service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:32.756887 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1062/2097] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:32 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:32.757081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1036/2098] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:32.758019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc38fe-e472-495e-9fe4-adfdf914ff6a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:32.773504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc38fe-e472-495e-9fe4-adfdf914ff6a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:32.781401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a49b6-885a-4243-bba8-95caa9713a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:32.782396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a49b6-885a-4243-bba8-95caa9713a9e admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:32.782851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a49b6-885a-4243-bba8-95caa9713a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:32.783687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a49b6-885a-4243-bba8-95caa9713a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:32.784166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a49b6-885a-4243-bba8-95caa9713a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:32.785963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a49b6-885a-4243-bba8-95caa9713a9e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:32.796723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1063/2099] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:32.801526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfe0dc-ff3e-42f0-ade2-4762e4d1daee service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:32.813616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfe0dc-ff3e-42f0-ade2-4762e4d1daee service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:32.818294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dca509-5c9d-4035-b69b-ab065b3108ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:32.819175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dca509-5c9d-4035-b69b-ab065b3108ab admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:32.819478 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dca509-5c9d-4035-b69b-ab065b3108ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:32.819737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dca509-5c9d-4035-b69b-ab065b3108ab admin admin] PATH_INFO: `/v3/projects/3bae3e36253c4e47b7b507f53663186b/users/68fab9785f514c5e9e45a0ec95d97604/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:32.835846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dca509-5c9d-4035-b69b-ab065b3108ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bae3e36253c4e47b7b507f53663186b, user_id=68fab9785f514c5e9e45a0ec95d97604, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:32.838461 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dca509-5c9d-4035-b69b-ab065b3108ab admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:32.928793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1037/2100] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:32 2022] PUT /identity/v3/projects/3bae3e36253c4e47b7b507f53663186b/users/68fab9785f514c5e9e45a0ec95d97604/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:32.933494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a49b6-885a-4243-bba8-95caa9713a9e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:32.943390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a49b6-885a-4243-bba8-95caa9713a9e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:33.010790 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c885a-6ba9-4a15-9cdc-8b4ac2305781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:33.012263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c885a-6ba9-4a15-9cdc-8b4ac2305781 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:33.012855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c885a-6ba9-4a15-9cdc-8b4ac2305781 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:33.013208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c885a-6ba9-4a15-9cdc-8b4ac2305781 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:33.014164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c885a-6ba9-4a15-9cdc-8b4ac2305781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:33.016336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6c885a-6ba9-4a15-9cdc-8b4ac2305781 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:33.032087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dca509-5c9d-4035-b69b-ab065b3108ab admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:33.033025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1064/2101] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:33.048188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dca509-5c9d-4035-b69b-ab065b3108ab admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:33.053161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4c5d8-0b99-482c-9ebc-c0fbe315228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:33.054173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4c5d8-0b99-482c-9ebc-c0fbe315228d admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:33.054527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4c5d8-0b99-482c-9ebc-c0fbe315228d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:33.054876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4c5d8-0b99-482c-9ebc-c0fbe315228d admin admin] PATH_INFO: `/v3/projects/3bae3e36253c4e47b7b507f53663186b/users/68fab9785f514c5e9e45a0ec95d97604/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:33.057020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4c5d8-0b99-482c-9ebc-c0fbe315228d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bae3e36253c4e47b7b507f53663186b, user_id=68fab9785f514c5e9e45a0ec95d97604, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:33.060515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4c5d8-0b99-482c-9ebc-c0fbe315228d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:33.072631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1038/2102] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:33 2022] PUT /identity/v3/projects/3bae3e36253c4e47b7b507f53663186b/users/68fab9785f514c5e9e45a0ec95d97604/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:33.244585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c885a-6ba9-4a15-9cdc-8b4ac2305781 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:33.247398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6d6c885a-6ba9-4a15-9cdc-8b4ac2305781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:33.262781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c885a-6ba9-4a15-9cdc-8b4ac2305781 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:33.273641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:33.335548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:33.335548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:33.335548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:33.335548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] PATH_INFO: `/v3/projects/4680a5d0b1024125b2ade61d7fc941b9` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:33.335900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4680a5d0b1024125b2ade61d7fc941b9)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:33.335900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:33.486550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4680a5d0b1024125b2ade61d7fc941b9', 'request_id': 'req-ec7319dc-1119-476d-9aa3-35b0dddf2760'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:33.556830 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4680a5d0b1024125b2ade61d7fc941b9', 'request_id': 'req-ec7319dc-1119-476d-9aa3-35b0dddf2760'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:33.563114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4680a5d0b1024125b2ade61d7fc941b9', 'request_id': 'req-ec7319dc-1119-476d-9aa3-35b0dddf2760'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:33.575442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] The token cache is being invalidate because project 4680a5d0b1024125b2ade61d7fc941b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:33.575732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:33.576389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:33.577983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1065/2103] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:33 2022] DELETE /identity/v3/projects/4680a5d0b1024125b2ade61d7fc941b9 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:33.582092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4c5d8-0b99-482c-9ebc-c0fbe315228d admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:33.584115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b9c4c5d8-0b99-482c-9ebc-c0fbe315228d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:33.597840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4c5d8-0b99-482c-9ebc-c0fbe315228d admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:33.669071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:33.670742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:33.677019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:33.677019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] PATH_INFO: `/v3/users/79ccdbb37323432499444ffab6cd2e09` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:33.677019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ccdbb37323432499444ffab6cd2e09)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:33.677019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:33.916466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ccdbb37323432499444ffab6cd2e09'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:33.928319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ccdbb37323432499444ffab6cd2e09'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:33.929058 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ccdbb37323432499444ffab6cd2e09'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:33.956960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ccdbb37323432499444ffab6cd2e09'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:33.958411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ccdbb37323432499444ffab6cd2e09'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:34.140560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1039/2104] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:33 2022] DELETE /identity/v3/users/79ccdbb37323432499444ffab6cd2e09 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:34.654604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:34.666942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7319dc-1119-476d-9aa3-35b0dddf2760 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:34.794705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:34.801891 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:34.802228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:34.803827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] PATH_INFO: `/v3/projects/2a57fbb665b74610881ccb35f6095ce6` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:34.808171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a57fbb665b74610881ccb35f6095ce6)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:34.811684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:34.919506 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:34.925446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a57fbb665b74610881ccb35f6095ce6', 'request_id': 'req-11adb960-fd4d-4b39-a0f2-0f99ba88952e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:35.021278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a57fbb665b74610881ccb35f6095ce6', 'request_id': 'req-11adb960-fd4d-4b39-a0f2-0f99ba88952e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:35.030978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a57fbb665b74610881ccb35f6095ce6', 'request_id': 'req-11adb960-fd4d-4b39-a0f2-0f99ba88952e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:35.047051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] The token cache is being invalidate because project 2a57fbb665b74610881ccb35f6095ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:35.047051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:35.048170 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:35.051065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1066/2105] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:34 2022] DELETE /identity/v3/projects/2a57fbb665b74610881ccb35f6095ce6 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:35.056661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:35.059364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:35.075757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87130d3d-fb60-40f3-b8e7-a6f9047b3d4a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:35.107538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:35.147470 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b04b301-a055-415a-aae1-af09e14b26c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:35.148658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04b301-a055-415a-aae1-af09e14b26c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:35.149171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04b301-a055-415a-aae1-af09e14b26c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:35.149731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04b301-a055-415a-aae1-af09e14b26c6 admin admin] PATH_INFO: `/v3/users/52c51d547f1d4ca5be20d7565dc5a511` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:35.152122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04b301-a055-415a-aae1-af09e14b26c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c51d547f1d4ca5be20d7565dc5a511)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:35.155148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b04b301-a055-415a-aae1-af09e14b26c6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:35.421857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4b04b301-a055-415a-aae1-af09e14b26c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c51d547f1d4ca5be20d7565dc5a511'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:35.441699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4b04b301-a055-415a-aae1-af09e14b26c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c51d547f1d4ca5be20d7565dc5a511'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:35.444952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4b04b301-a055-415a-aae1-af09e14b26c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c51d547f1d4ca5be20d7565dc5a511'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:35.491523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4b04b301-a055-415a-aae1-af09e14b26c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c51d547f1d4ca5be20d7565dc5a511'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:35.492297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4b04b301-a055-415a-aae1-af09e14b26c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c51d547f1d4ca5be20d7565dc5a511'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:35.639293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1040/2106] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:35 2022] DELETE /identity/v3/users/52c51d547f1d4ca5be20d7565dc5a511 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:36.462342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:36.465613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:36.484173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11adb960-fd4d-4b39-a0f2-0f99ba88952e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:36.583893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:36.585963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:36.586255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:36.586583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] PATH_INFO: `/v3/projects/fa7a11a257c242b0bfcfe7cfb1db6157` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:36.587903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa7a11a257c242b0bfcfe7cfb1db6157)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:36.590538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:36.760911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa7a11a257c242b0bfcfe7cfb1db6157', 'request_id': 'req-1ac377c1-0d69-4ea7-8c49-160ef44a349c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:36.834187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa7a11a257c242b0bfcfe7cfb1db6157', 'request_id': 'req-1ac377c1-0d69-4ea7-8c49-160ef44a349c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:36.842572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa7a11a257c242b0bfcfe7cfb1db6157', 'request_id': 'req-1ac377c1-0d69-4ea7-8c49-160ef44a349c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:36.860853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] The token cache is being invalidate because project fa7a11a257c242b0bfcfe7cfb1db6157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:36.861155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:36.861998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1ac377c1-0d69-4ea7-8c49-160ef44a349c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:36.864822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1067/2107] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:36 2022] DELETE /identity/v3/projects/fa7a11a257c242b0bfcfe7cfb1db6157 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:36.870676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ef402-f554-43f5-b682-e199f0237a42 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:36.871187 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ef402-f554-43f5-b682-e199f0237a42 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:36.871265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ef402-f554-43f5-b682-e199f0237a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:36.913829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-730ef402-f554-43f5-b682-e199f0237a42 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:36.915308 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-730ef402-f554-43f5-b682-e199f0237a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:37.020790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1041/2108] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:37.029717 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9432bda-8822-433c-bc0b-f5d4cf6e6399 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:37.030047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9432bda-8822-433c-bc0b-f5d4cf6e6399 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:37.030453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9432bda-8822-433c-bc0b-f5d4cf6e6399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:37.059248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-a9432bda-8822-433c-bc0b-f5d4cf6e6399 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:37.061811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a9432bda-8822-433c-bc0b-f5d4cf6e6399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:37.071601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1068/2109] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:37.076303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ef402-f554-43f5-b682-e199f0237a42 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:37.087663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ef402-f554-43f5-b682-e199f0237a42 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:37.092892 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c73df-1763-430a-bcd2-21a5230a8275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:37.093608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c73df-1763-430a-bcd2-21a5230a8275 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:37.093895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c73df-1763-430a-bcd2-21a5230a8275 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:37.094323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c73df-1763-430a-bcd2-21a5230a8275 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:37.095234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c73df-1763-430a-bcd2-21a5230a8275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:37.095699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c73df-1763-430a-bcd2-21a5230a8275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:37.097648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c73df-1763-430a-bcd2-21a5230a8275 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:37.116268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1042/2110] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:37.117158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9432bda-8822-433c-bc0b-f5d4cf6e6399 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:37.129485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9432bda-8822-433c-bc0b-f5d4cf6e6399 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:37.135509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11179fe3-450e-40c9-9ef8-2fc43417b1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:37.136289 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11179fe3-450e-40c9-9ef8-2fc43417b1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:37.136549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11179fe3-450e-40c9-9ef8-2fc43417b1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:37.136962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11179fe3-450e-40c9-9ef8-2fc43417b1b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:37.137756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11179fe3-450e-40c9-9ef8-2fc43417b1b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:37.139753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11179fe3-450e-40c9-9ef8-2fc43417b1b4 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:37.263286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1069/2111] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:37 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:37.267023 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c73df-1763-430a-bcd2-21a5230a8275 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:37.277810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c73df-1763-430a-bcd2-21a5230a8275 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:37.312392 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:37.354045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc4c6-9ad2-4e19-a24c-3853b9a8a45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:37.355543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc4c6-9ad2-4e19-a24c-3853b9a8a45a admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:37.355830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc4c6-9ad2-4e19-a24c-3853b9a8a45a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:37.356099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc4c6-9ad2-4e19-a24c-3853b9a8a45a admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:37.356795 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbc4c6-9ad2-4e19-a24c-3853b9a8a45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:37.358762 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbc4c6-9ad2-4e19-a24c-3853b9a8a45a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:37.455780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1043/2112] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:37 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:37.459686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11179fe3-450e-40c9-9ef8-2fc43417b1b4 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:37.472165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11179fe3-450e-40c9-9ef8-2fc43417b1b4 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:37.478450 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce423f0d-5202-4e14-a877-71e5c7efee34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:37.479260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce423f0d-5202-4e14-a877-71e5c7efee34 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:37.479670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce423f0d-5202-4e14-a877-71e5c7efee34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:37.480150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce423f0d-5202-4e14-a877-71e5c7efee34 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:37.481807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce423f0d-5202-4e14-a877-71e5c7efee34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:37.483716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce423f0d-5202-4e14-a877-71e5c7efee34 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:37.502038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1070/2113] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:37.503783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc4c6-9ad2-4e19-a24c-3853b9a8a45a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:37.516661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc4c6-9ad2-4e19-a24c-3853b9a8a45a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:37.526437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93016a-d37e-4ab8-b160-e3890cadd202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:37.527270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93016a-d37e-4ab8-b160-e3890cadd202 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:37.528877 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93016a-d37e-4ab8-b160-e3890cadd202 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:37.529385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93016a-d37e-4ab8-b160-e3890cadd202 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:37.530813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93016a-d37e-4ab8-b160-e3890cadd202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:37.532670 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93016a-d37e-4ab8-b160-e3890cadd202 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:37.551176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1044/2114] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:37.555316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce423f0d-5202-4e14-a877-71e5c7efee34 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:37.565895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce423f0d-5202-4e14-a877-71e5c7efee34 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:37.570562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d06b32-4fac-497f-ade5-b5a5c876516d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:37.587597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d06b32-4fac-497f-ade5-b5a5c876516d admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:37.587597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d06b32-4fac-497f-ade5-b5a5c876516d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:37.587597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d06b32-4fac-497f-ade5-b5a5c876516d admin admin] PATH_INFO: `/v3/projects/25c8cef61e144ea6a0b6ed80e00d6f68/users/b901e435966247e0b162b2be3d58e7dd/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:37.587597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d06b32-4fac-497f-ade5-b5a5c876516d admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c8cef61e144ea6a0b6ed80e00d6f68, user_id=b901e435966247e0b162b2be3d58e7dd, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:37.590729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d06b32-4fac-497f-ade5-b5a5c876516d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:37.672806 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:37.681793 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1071/2115] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:37 2022] PUT /identity/v3/projects/25c8cef61e144ea6a0b6ed80e00d6f68/users/b901e435966247e0b162b2be3d58e7dd/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:37.694779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2766d10-1f55-492e-9150-9c359debd4f9 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:37.695064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2766d10-1f55-492e-9150-9c359debd4f9 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:37.695347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2766d10-1f55-492e-9150-9c359debd4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:37.760044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-c2766d10-1f55-492e-9150-9c359debd4f9 None None] MFA Rules not processed for user `b901e435966247e0b162b2be3d58e7dd`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:37.761926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c2766d10-1f55-492e-9150-9c359debd4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:37.837072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1045/2116] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:37.842038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d06b32-4fac-497f-ade5-b5a5c876516d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:37.925657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d06b32-4fac-497f-ade5-b5a5c876516d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:37.930432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52c1ff-bed0-4484-a2a3-f62e88116e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:37.931235 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52c1ff-bed0-4484-a2a3-f62e88116e3a admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:37.931524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52c1ff-bed0-4484-a2a3-f62e88116e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:37.931801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b52c1ff-bed0-4484-a2a3-f62e88116e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:37.933260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52c1ff-bed0-4484-a2a3-f62e88116e3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:37.935195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b52c1ff-bed0-4484-a2a3-f62e88116e3a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:38.047082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1072/2117] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:37 2022] POST /identity/v3/projects => generated 387 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:38.050834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2766d10-1f55-492e-9150-9c359debd4f9 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:38.067980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2766d10-1f55-492e-9150-9c359debd4f9 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:38.142412 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0164c5-1870-4b29-91f5-fc6b4726d784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:38.143111 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0164c5-1870-4b29-91f5-fc6b4726d784 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:38.145453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0164c5-1870-4b29-91f5-fc6b4726d784 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:38.146030 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0164c5-1870-4b29-91f5-fc6b4726d784 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:38.146973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0164c5-1870-4b29-91f5-fc6b4726d784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:38.150883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0164c5-1870-4b29-91f5-fc6b4726d784 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:38.256616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1046/2118] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:38 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:38.261777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52c1ff-bed0-4484-a2a3-f62e88116e3a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:38.272399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b52c1ff-bed0-4484-a2a3-f62e88116e3a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:38.276983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1407e-901a-4db7-85a5-5702b2ad6611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:38.277778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1407e-901a-4db7-85a5-5702b2ad6611 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:38.278069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1407e-901a-4db7-85a5-5702b2ad6611 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:38.278340 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1407e-901a-4db7-85a5-5702b2ad6611 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:38.279069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1407e-901a-4db7-85a5-5702b2ad6611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:38.280749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1407e-901a-4db7-85a5-5702b2ad6611 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:38.294514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1073/2119] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:38.297746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0164c5-1870-4b29-91f5-fc6b4726d784 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:38.310002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0164c5-1870-4b29-91f5-fc6b4726d784 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:38.314488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c102349-8cd9-4e38-af75-b350b4899a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:38.315424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c102349-8cd9-4e38-af75-b350b4899a8c admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:38.316722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c102349-8cd9-4e38-af75-b350b4899a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:38.316722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c102349-8cd9-4e38-af75-b350b4899a8c admin admin] PATH_INFO: `/v3/projects/0c5e970449d34a0eb973b342a81a05cc/users/03305455bdae4f8ea311f582b16642f6/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:38.332870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c102349-8cd9-4e38-af75-b350b4899a8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5e970449d34a0eb973b342a81a05cc, user_id=03305455bdae4f8ea311f582b16642f6, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:38.335572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c102349-8cd9-4e38-af75-b350b4899a8c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:38.417593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1407e-901a-4db7-85a5-5702b2ad6611 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:38.420046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1047/2120] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:38 2022] PUT /identity/v3/projects/0c5e970449d34a0eb973b342a81a05cc/users/03305455bdae4f8ea311f582b16642f6/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:38.428272 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1407e-901a-4db7-85a5-5702b2ad6611 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:38.494840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce03c5-1b7e-4ef3-81de-6252fdd0ceae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:38.496128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce03c5-1b7e-4ef3-81de-6252fdd0ceae admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:38.496454 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce03c5-1b7e-4ef3-81de-6252fdd0ceae admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:38.496700 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce03c5-1b7e-4ef3-81de-6252fdd0ceae admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:38.497890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce03c5-1b7e-4ef3-81de-6252fdd0ceae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:38.499186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcce03c5-1b7e-4ef3-81de-6252fdd0ceae admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:38.510636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1074/2121] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:38.518131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c102349-8cd9-4e38-af75-b350b4899a8c admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:38.532593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c102349-8cd9-4e38-af75-b350b4899a8c admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:38.537597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd89df-ff09-4b8e-ba09-0efe9a444824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:38.538635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd89df-ff09-4b8e-ba09-0efe9a444824 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:38.539096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd89df-ff09-4b8e-ba09-0efe9a444824 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:38.539460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd89df-ff09-4b8e-ba09-0efe9a444824 admin admin] PATH_INFO: `/v3/projects/0c5e970449d34a0eb973b342a81a05cc/users/03305455bdae4f8ea311f582b16642f6/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:38.541621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd89df-ff09-4b8e-ba09-0efe9a444824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5e970449d34a0eb973b342a81a05cc, user_id=03305455bdae4f8ea311f582b16642f6, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:38.544411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd89df-ff09-4b8e-ba09-0efe9a444824 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:38.546959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:38.556917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1048/2122] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:38 2022] PUT /identity/v3/projects/0c5e970449d34a0eb973b342a81a05cc/users/03305455bdae4f8ea311f582b16642f6/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:38.567627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62124c96-48d9-42a1-9eff-5240f9d6e92a None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:38.567954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62124c96-48d9-42a1-9eff-5240f9d6e92a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:38.568333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62124c96-48d9-42a1-9eff-5240f9d6e92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:38.597475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-62124c96-48d9-42a1-9eff-5240f9d6e92a None None] MFA Rules not processed for user `03305455bdae4f8ea311f582b16642f6`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:38.598849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-62124c96-48d9-42a1-9eff-5240f9d6e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:38.679943 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1075/2123] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:38.694378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd89df-ff09-4b8e-ba09-0efe9a444824 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:38.694570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:38.699338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-3cfd89df-ff09-4b8e-ba09-0efe9a444824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:38.778190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd89df-ff09-4b8e-ba09-0efe9a444824 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:38.782106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77920f86-682d-418b-95e7-8bc8ccbae95f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:38.782887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77920f86-682d-418b-95e7-8bc8ccbae95f service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:38.783336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77920f86-682d-418b-95e7-8bc8ccbae95f service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:38.783826 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77920f86-682d-418b-95e7-8bc8ccbae95f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:38.794439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77920f86-682d-418b-95e7-8bc8ccbae95f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:38.795990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77920f86-682d-418b-95e7-8bc8ccbae95f service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:38.806109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1049/2124] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:38 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:39.959678 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f3957-f5fc-410d-ae4a-4601df8cecdf None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:39.960185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f3957-f5fc-410d-ae4a-4601df8cecdf None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:39.960299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f3957-f5fc-410d-ae4a-4601df8cecdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:40.002654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6a6f3957-f5fc-410d-ae4a-4601df8cecdf None None] MFA Rules not processed for user `68fab9785f514c5e9e45a0ec95d97604`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:40.004295 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6a6f3957-f5fc-410d-ae4a-4601df8cecdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:40.081230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1076/2125] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:39 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:40.099255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77920f86-682d-418b-95e7-8bc8ccbae95f service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:40.101692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-77920f86-682d-418b-95e7-8bc8ccbae95f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:40.111700 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f3957-f5fc-410d-ae4a-4601df8cecdf None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:40.212864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f3957-f5fc-410d-ae4a-4601df8cecdf None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:40.220700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77920f86-682d-418b-95e7-8bc8ccbae95f service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:40.224254 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:40.226637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:40.226930 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:40.227199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] PATH_INFO: `/v3/users/b901e435966247e0b162b2be3d58e7dd` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:40.228462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b901e435966247e0b162b2be3d58e7dd)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:40.231773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:40.239177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aafe63-db14-409b-a898-fe41c71d48c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:40.243205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aafe63-db14-409b-a898-fe41c71d48c0 service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:40.243205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aafe63-db14-409b-a898-fe41c71d48c0 service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:40.243205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aafe63-db14-409b-a898-fe41c71d48c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:40.243205 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a7aafe63-db14-409b-a898-fe41c71d48c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:40.327795 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aafe63-db14-409b-a898-fe41c71d48c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:40.329522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aafe63-db14-409b-a898-fe41c71d48c0 service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:40.354046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1050/2126] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:40.457983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b901e435966247e0b162b2be3d58e7dd'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:40.458866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b901e435966247e0b162b2be3d58e7dd'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:40.477691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b901e435966247e0b162b2be3d58e7dd'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:40.478881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b901e435966247e0b162b2be3d58e7dd'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:40.528141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b901e435966247e0b162b2be3d58e7dd'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:40.601157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:40.614157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1077/2127] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:40 2022] DELETE /identity/v3/users/b901e435966247e0b162b2be3d58e7dd => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:40.615894 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb3c36-0842-434b-9716-65217c6063e7 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:40.616134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb3c36-0842-434b-9716-65217c6063e7 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:40.616413 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb3c36-0842-434b-9716-65217c6063e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:40.645699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-93cb3c36-0842-434b-9716-65217c6063e7 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:40.646964 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-93cb3c36-0842-434b-9716-65217c6063e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:40.728946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:40.772848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1051/2128] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:40.783908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:40.785589 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:40.884979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4cc41-dbaf-4b50-9a74-70dbac4e7590 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:40.888864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8949e8-76c6-4ae0-a6dd-8b0dee786385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:40.889688 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8949e8-76c6-4ae0-a6dd-8b0dee786385 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:40.890041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8949e8-76c6-4ae0-a6dd-8b0dee786385 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:40.890384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8949e8-76c6-4ae0-a6dd-8b0dee786385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:40.917595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8949e8-76c6-4ae0-a6dd-8b0dee786385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:40.919963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8949e8-76c6-4ae0-a6dd-8b0dee786385 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:40.932192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1078/2129] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:41.383227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb3c36-0842-434b-9716-65217c6063e7 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:41.383227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-93cb3c36-0842-434b-9716-65217c6063e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:41.394919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb3c36-0842-434b-9716-65217c6063e7 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:41.399768 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3d2f4-9cf5-4b2b-9b26-3377ad6d0085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:41.400676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3d2f4-9cf5-4b2b-9b26-3377ad6d0085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:41.400972 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3d2f4-9cf5-4b2b-9b26-3377ad6d0085 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:41.401798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3d2f4-9cf5-4b2b-9b26-3377ad6d0085 admin admin] PATH_INFO: `/v3/users/85c4be4dc6104645a7662ce637444f35` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:41.406711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3d2f4-9cf5-4b2b-9b26-3377ad6d0085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85c4be4dc6104645a7662ce637444f35)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:41.406711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3d2f4-9cf5-4b2b-9b26-3377ad6d0085 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:41.410635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8949e8-76c6-4ae0-a6dd-8b0dee786385 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:41.412616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4e8949e8-76c6-4ae0-a6dd-8b0dee786385 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:41.429350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8949e8-76c6-4ae0-a6dd-8b0dee786385 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:41.434730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:41.438641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:41.438641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:41.438641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] PATH_INFO: `/v3/projects/25c8cef61e144ea6a0b6ed80e00d6f68` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:41.438641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c8cef61e144ea6a0b6ed80e00d6f68)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:41.439685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:41.626798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c7c3d2f4-9cf5-4b2b-9b26-3377ad6d0085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85c4be4dc6104645a7662ce637444f35'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:41.628799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c8cef61e144ea6a0b6ed80e00d6f68', 'request_id': 'req-36d85280-4292-41d6-87e0-65598bb4834f'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:41.636737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c7c3d2f4-9cf5-4b2b-9b26-3377ad6d0085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85c4be4dc6104645a7662ce637444f35'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:41.637694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c7c3d2f4-9cf5-4b2b-9b26-3377ad6d0085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85c4be4dc6104645a7662ce637444f35'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:41.650508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c7c3d2f4-9cf5-4b2b-9b26-3377ad6d0085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85c4be4dc6104645a7662ce637444f35'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:41.651793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c7c3d2f4-9cf5-4b2b-9b26-3377ad6d0085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85c4be4dc6104645a7662ce637444f35'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:41.742791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c8cef61e144ea6a0b6ed80e00d6f68', 'request_id': 'req-36d85280-4292-41d6-87e0-65598bb4834f'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:41.754159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c8cef61e144ea6a0b6ed80e00d6f68', 'request_id': 'req-36d85280-4292-41d6-87e0-65598bb4834f'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:41.795077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] The token cache is being invalidate because project 25c8cef61e144ea6a0b6ed80e00d6f68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:41.795937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:41.801842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:41.801842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1079/2130] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:41 2022] DELETE /identity/v3/projects/25c8cef61e144ea6a0b6ed80e00d6f68 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:41.804631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:41.809531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:41.827532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1052/2131] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:41 2022] DELETE /identity/v3/users/85c4be4dc6104645a7662ce637444f35 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:41.833405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76088fc8-b933-413b-be3d-19cb4acc29b9 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:41.833736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76088fc8-b933-413b-be3d-19cb4acc29b9 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:41.834189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76088fc8-b933-413b-be3d-19cb4acc29b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:41.856770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-76088fc8-b933-413b-be3d-19cb4acc29b9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:41.859094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-76088fc8-b933-413b-be3d-19cb4acc29b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:41.907408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d85280-4292-41d6-87e0-65598bb4834f admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:41.943132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:41.950008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1053/2132] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:41.962234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76088fc8-b933-413b-be3d-19cb4acc29b9 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:41.967139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-76088fc8-b933-413b-be3d-19cb4acc29b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:41.995826 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:41.999759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:42.000058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:42.000341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] PATH_INFO: `/v3/users/03305455bdae4f8ea311f582b16642f6` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:42.006978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03305455bdae4f8ea311f582b16642f6)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:42.010074 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:42.051486 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:42.056793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76088fc8-b933-413b-be3d-19cb4acc29b9 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:42.061046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b51704-fb31-4f0a-a54e-be813c269f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:42.061828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b51704-fb31-4f0a-a54e-be813c269f0f service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:42.062118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b51704-fb31-4f0a-a54e-be813c269f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:42.062399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b51704-fb31-4f0a-a54e-be813c269f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:42.071542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b51704-fb31-4f0a-a54e-be813c269f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:42.073024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b51704-fb31-4f0a-a54e-be813c269f0f service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:42.083072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1054/2133] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:42.221734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03305455bdae4f8ea311f582b16642f6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:42.222392 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03305455bdae4f8ea311f582b16642f6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:42.229358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03305455bdae4f8ea311f582b16642f6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:42.231419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03305455bdae4f8ea311f582b16642f6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:42.256114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03305455bdae4f8ea311f582b16642f6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:42.381273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1080/2134] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:41 2022] DELETE /identity/v3/users/03305455bdae4f8ea311f582b16642f6 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:42.613626 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b51704-fb31-4f0a-a54e-be813c269f0f service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:42.613626 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-13b51704-fb31-4f0a-a54e-be813c269f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:42.627961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b51704-fb31-4f0a-a54e-be813c269f0f service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:42.693147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:42.694110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:42.694898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:42.694898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] PATH_INFO: `/v3/projects/d135af04e85245efab72dc1f1f1cd245` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:42.696833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d135af04e85245efab72dc1f1f1cd245)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:42.701799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:42.867445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd135af04e85245efab72dc1f1f1cd245', 'request_id': 'req-d96bae9a-0054-4bb5-b580-e2ff4827bbec'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:42.919460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd135af04e85245efab72dc1f1f1cd245', 'request_id': 'req-d96bae9a-0054-4bb5-b580-e2ff4827bbec'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:42.930051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:42.932218 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:42.952275 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd135af04e85245efab72dc1f1f1cd245', 'request_id': 'req-d96bae9a-0054-4bb5-b580-e2ff4827bbec'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:42.955077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1c9377-2e8a-4532-be3a-93905d67d732 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:42.983107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] The token cache is being invalidate because project d135af04e85245efab72dc1f1f1cd245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:42.983921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:42.986965 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d96bae9a-0054-4bb5-b580-e2ff4827bbec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:42.990116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1055/2135] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:42 2022] DELETE /identity/v3/projects/d135af04e85245efab72dc1f1f1cd245 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:42.998883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ef9a7-762f-4c5c-92ea-9b0c21dbf6ab None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:42.999186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ef9a7-762f-4c5c-92ea-9b0c21dbf6ab None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:42.999459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ef9a7-762f-4c5c-92ea-9b0c21dbf6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.041230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-226ef9a7-762f-4c5c-92ea-9b0c21dbf6ab None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:43.042522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-226ef9a7-762f-4c5c-92ea-9b0c21dbf6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:43.069636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:43.070556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.070849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.071134 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] PATH_INFO: `/v3/projects/0c5e970449d34a0eb973b342a81a05cc` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.072478 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5e970449d34a0eb973b342a81a05cc)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:43.080540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:43.134735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1056/2136] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:43.142695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99ecac-3475-4e34-9d7d-bf87ce9feef8 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.143113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99ecac-3475-4e34-9d7d-bf87ce9feef8 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.143469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99ecac-3475-4e34-9d7d-bf87ce9feef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.161778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-fa99ecac-3475-4e34-9d7d-bf87ce9feef8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:43.162929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-fa99ecac-3475-4e34-9d7d-bf87ce9feef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:43.173440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1057/2137] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:43.177204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99ecac-3475-4e34-9d7d-bf87ce9feef8 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:43.188279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99ecac-3475-4e34-9d7d-bf87ce9feef8 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:43.193784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce0617-2415-47ec-a33a-13b048cfd494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:43.194551 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce0617-2415-47ec-a33a-13b048cfd494 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.194938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce0617-2415-47ec-a33a-13b048cfd494 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.195587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce0617-2415-47ec-a33a-13b048cfd494 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.196149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce0617-2415-47ec-a33a-13b048cfd494 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:43.196524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce0617-2415-47ec-a33a-13b048cfd494 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:43.198489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce0617-2415-47ec-a33a-13b048cfd494 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:43.209407 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:43.223931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1058/2138] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:57:43.226170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce0617-2415-47ec-a33a-13b048cfd494 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:43.239586 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5e970449d34a0eb973b342a81a05cc', 'request_id': 'req-be2e499c-8793-49aa-8cd2-2c3f5040895e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:43.239706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce0617-2415-47ec-a33a-13b048cfd494 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:43.239924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:43.294565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5e970449d34a0eb973b342a81a05cc', 'request_id': 'req-be2e499c-8793-49aa-8cd2-2c3f5040895e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:43.295774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531be07e-9e1d-467f-ad1c-3e7e371c127f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:43.296448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531be07e-9e1d-467f-ad1c-3e7e371c127f admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.296864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531be07e-9e1d-467f-ad1c-3e7e371c127f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.297151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531be07e-9e1d-467f-ad1c-3e7e371c127f admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.298516 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531be07e-9e1d-467f-ad1c-3e7e371c127f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:43.299997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531be07e-9e1d-467f-ad1c-3e7e371c127f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:43.328761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5e970449d34a0eb973b342a81a05cc', 'request_id': 'req-be2e499c-8793-49aa-8cd2-2c3f5040895e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:43.347950 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] The token cache is being invalidate because project 0c5e970449d34a0eb973b342a81a05cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:43.348247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:43.348932 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-be2e499c-8793-49aa-8cd2-2c3f5040895e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:43.350902 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1081/2139] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:42 2022] DELETE /identity/v3/projects/0c5e970449d34a0eb973b342a81a05cc => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:43.352647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1ca06-d275-4689-8e43-b848c9511616 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.353074 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1ca06-d275-4689-8e43-b848c9511616 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.353398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1ca06-d275-4689-8e43-b848c9511616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.377725 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-11f1ca06-d275-4689-8e43-b848c9511616 None None] MFA Rules not processed for user `0569765b6f07473fa07a5376b3677da8`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:43.382198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-11f1ca06-d275-4689-8e43-b848c9511616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:43.415249 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1059/2140] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:43 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:43.417714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408c220-0913-4031-acc7-1c4c3b82d21b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.421338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408c220-0913-4031-acc7-1c4c3b82d21b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.421338 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408c220-0913-4031-acc7-1c4c3b82d21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.438723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-c408c220-0913-4031-acc7-1c4c3b82d21b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:43.440017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c408c220-0913-4031-acc7-1c4c3b82d21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:43.502319 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1082/2141] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:43.506485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1ca06-d275-4689-8e43-b848c9511616 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:43.509990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-11f1ca06-d275-4689-8e43-b848c9511616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:43.530145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1060/2142] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:43.537958 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cffe9-36f0-4e70-b387-c51613ca9f49 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.538254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cffe9-36f0-4e70-b387-c51613ca9f49 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.538535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cffe9-36f0-4e70-b387-c51613ca9f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.541787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1ca06-d275-4689-8e43-b848c9511616 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:43.553925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e9f0-0f1a-4a49-a69e-815bcbd97a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:43.555032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e9f0-0f1a-4a49-a69e-815bcbd97a97 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.555032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e9f0-0f1a-4a49-a69e-815bcbd97a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.555207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7309e9f0-0f1a-4a49-a69e-815bcbd97a97 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.556243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309e9f0-0f1a-4a49-a69e-815bcbd97a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:43.562364 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7309e9f0-0f1a-4a49-a69e-815bcbd97a97 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:43.577008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ba8cffe9-36f0-4e70-b387-c51613ca9f49 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:43.578253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ba8cffe9-36f0-4e70-b387-c51613ca9f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:43.593584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1061/2143] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:43.598278 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cffe9-36f0-4e70-b387-c51613ca9f49 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:43.610292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cffe9-36f0-4e70-b387-c51613ca9f49 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:43.616565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a774-a4dd-4741-8823-98016dc851d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:43.617274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a774-a4dd-4741-8823-98016dc851d6 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.617584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a774-a4dd-4741-8823-98016dc851d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.618095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a774-a4dd-4741-8823-98016dc851d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.618716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8a774-a4dd-4741-8823-98016dc851d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:43.619041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8a774-a4dd-4741-8823-98016dc851d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:43.621654 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8a774-a4dd-4741-8823-98016dc851d6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:43.641290 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1062/2144] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:43.643790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a774-a4dd-4741-8823-98016dc851d6 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:43.670619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a774-a4dd-4741-8823-98016dc851d6 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:43.684111 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59b6bd-7280-428b-93fa-9212717167d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:43.684880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59b6bd-7280-428b-93fa-9212717167d4 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.685231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59b6bd-7280-428b-93fa-9212717167d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.685618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59b6bd-7280-428b-93fa-9212717167d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.706107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59b6bd-7280-428b-93fa-9212717167d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:43.710780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59b6bd-7280-428b-93fa-9212717167d4 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:43.714191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1083/2145] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:43 2022] POST /identity/v3/users => generated 286 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:43.715841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e9f0-0f1a-4a49-a69e-815bcbd97a97 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:43.747763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7309e9f0-0f1a-4a49-a69e-815bcbd97a97 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:43.756069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b739f83-d1a4-4ae0-85f2-92af7fc04ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:43.757724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b739f83-d1a4-4ae0-85f2-92af7fc04ccc admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.768259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b739f83-d1a4-4ae0-85f2-92af7fc04ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.768259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b739f83-d1a4-4ae0-85f2-92af7fc04ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.768860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b739f83-d1a4-4ae0-85f2-92af7fc04ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:43.773755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b739f83-d1a4-4ae0-85f2-92af7fc04ccc admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:43.789329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1084/2146] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:43.790316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b739f83-d1a4-4ae0-85f2-92af7fc04ccc admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:43.811804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b739f83-d1a4-4ae0-85f2-92af7fc04ccc admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:43.825530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538fa78-5a2e-4564-aedf-385b4b2b3f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:43.826428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538fa78-5a2e-4564-aedf-385b4b2b3f76 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.831995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1063/2147] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:43 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:43.836395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59b6bd-7280-428b-93fa-9212717167d4 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:43.836815 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538fa78-5a2e-4564-aedf-385b4b2b3f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.837144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538fa78-5a2e-4564-aedf-385b4b2b3f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.839200 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538fa78-5a2e-4564-aedf-385b4b2b3f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:43.851159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538fa78-5a2e-4564-aedf-385b4b2b3f76 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:43.873346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1085/2148] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:43.880423 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538fa78-5a2e-4564-aedf-385b4b2b3f76 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:43.908190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538fa78-5a2e-4564-aedf-385b4b2b3f76 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:43.970370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59b6bd-7280-428b-93fa-9212717167d4 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:43.983920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3f594-edef-4b67-8435-d71d983b9695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:43.984541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3f594-edef-4b67-8435-d71d983b9695 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:43.984827 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3f594-edef-4b67-8435-d71d983b9695 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:43.992180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3f594-edef-4b67-8435-d71d983b9695 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:43.992180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3f594-edef-4b67-8435-d71d983b9695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:43.992180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3f594-edef-4b67-8435-d71d983b9695 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:44.009366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:44.036739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3abb8a-ef67-49dc-be37-f7b898526ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:44.042681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3abb8a-ef67-49dc-be37-f7b898526ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:44.042962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3abb8a-ef67-49dc-be37-f7b898526ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:44.045784 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3abb8a-ef67-49dc-be37-f7b898526ac2 admin admin] PATH_INFO: `/v3/projects/0b2bae632b624df0a2bedfad6548e423/users/4c35fa0c8e3b42efbe30af614b4c6db1/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:44.084756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3abb8a-ef67-49dc-be37-f7b898526ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2bae632b624df0a2bedfad6548e423, user_id=4c35fa0c8e3b42efbe30af614b4c6db1, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:44.087709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3abb8a-ef67-49dc-be37-f7b898526ac2 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:44.156478 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1064/2149] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:43 2022] POST /identity/v3/users => generated 290 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:44.157618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3f594-edef-4b67-8435-d71d983b9695 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:44.168561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3f594-edef-4b67-8435-d71d983b9695 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:44.174114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad30d0-7f6b-4c16-8312-6e624df19e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:44.175062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad30d0-7f6b-4c16-8312-6e624df19e77 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:44.175584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad30d0-7f6b-4c16-8312-6e624df19e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:44.176298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad30d0-7f6b-4c16-8312-6e624df19e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:44.177116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad30d0-7f6b-4c16-8312-6e624df19e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:44.179195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad30d0-7f6b-4c16-8312-6e624df19e77 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:44.182043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1086/2150] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:43 2022] PUT /identity/v3/projects/0b2bae632b624df0a2bedfad6548e423/users/4c35fa0c8e3b42efbe30af614b4c6db1/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:44.193718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072c6f8-e651-414e-af94-72fb5aa544ed None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:44.199180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072c6f8-e651-414e-af94-72fb5aa544ed None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:44.200363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072c6f8-e651-414e-af94-72fb5aa544ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:44.208093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1065/2151] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:44.209629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad30d0-7f6b-4c16-8312-6e624df19e77 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:44.226984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad30d0-7f6b-4c16-8312-6e624df19e77 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:44.255209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9072c6f8-e651-414e-af94-72fb5aa544ed None None] MFA Rules not processed for user `4c35fa0c8e3b42efbe30af614b4c6db1`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:44.256801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9072c6f8-e651-414e-af94-72fb5aa544ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:44.286351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:44.336799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e95d0-9d52-4b8b-92bc-4c455577e40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:44.337827 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e95d0-9d52-4b8b-92bc-4c455577e40a admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:44.338231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e95d0-9d52-4b8b-92bc-4c455577e40a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:44.338585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e95d0-9d52-4b8b-92bc-4c455577e40a admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:44.349297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e95d0-9d52-4b8b-92bc-4c455577e40a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:44.351037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e95d0-9d52-4b8b-92bc-4c455577e40a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:44.363804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1066/2152] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:44.378475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1087/2153] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:44.381653 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e95d0-9d52-4b8b-92bc-4c455577e40a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:44.390228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072c6f8-e651-414e-af94-72fb5aa544ed None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:44.392054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9072c6f8-e651-414e-af94-72fb5aa544ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:44.415628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e95d0-9d52-4b8b-92bc-4c455577e40a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:44.429969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5e24e-3fdf-454e-8373-612c079e6d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:44.430940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5e24e-3fdf-454e-8373-612c079e6d9e admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:44.431296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5e24e-3fdf-454e-8373-612c079e6d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:44.431585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5e24e-3fdf-454e-8373-612c079e6d9e admin admin] PATH_INFO: `/v3/projects/a130a94e28254c33a6524ed44916173f/users/5d3a20ade4eb484dbdba11f0213a52d7/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:44.464454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5e24e-3fdf-454e-8373-612c079e6d9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a130a94e28254c33a6524ed44916173f, user_id=5d3a20ade4eb484dbdba11f0213a52d7, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:44.464798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5e24e-3fdf-454e-8373-612c079e6d9e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:44.518037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072c6f8-e651-414e-af94-72fb5aa544ed None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:44.524209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48e49-2876-484b-8380-84b4bbd12537 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:44.524874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48e49-2876-484b-8380-84b4bbd12537 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:44.525164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48e49-2876-484b-8380-84b4bbd12537 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:44.525495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48e49-2876-484b-8380-84b4bbd12537 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:44.537604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb48e49-2876-484b-8380-84b4bbd12537 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:44.539087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb48e49-2876-484b-8380-84b4bbd12537 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:44.549403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1067/2154] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:44 2022] PUT /identity/v3/projects/a130a94e28254c33a6524ed44916173f/users/5d3a20ade4eb484dbdba11f0213a52d7/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:44.550051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1088/2155] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:44 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:57:44.562627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5be604-b44c-4f51-9563-6746949bcbf7 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:44.562921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5be604-b44c-4f51-9563-6746949bcbf7 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:44.563202 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5be604-b44c-4f51-9563-6746949bcbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:44.648441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-bd5be604-b44c-4f51-9563-6746949bcbf7 None None] MFA Rules not processed for user `5d3a20ade4eb484dbdba11f0213a52d7`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:44.649920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-bd5be604-b44c-4f51-9563-6746949bcbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:44.797509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1068/2156] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:44.800065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48e49-2876-484b-8380-84b4bbd12537 service nova] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:44.820225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48e49-2876-484b-8380-84b4bbd12537 service nova] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:44.931028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55f170-006e-4553-9c66-fe8cad9fe2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:44.931768 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55f170-006e-4553-9c66-fe8cad9fe2cb admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:44.932140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55f170-006e-4553-9c66-fe8cad9fe2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:44.932576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55f170-006e-4553-9c66-fe8cad9fe2cb admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:44.933512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55f170-006e-4553-9c66-fe8cad9fe2cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:44.936668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55f170-006e-4553-9c66-fe8cad9fe2cb admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:45.059551 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1089/2157] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:44 2022] POST /identity/v3/projects => generated 381 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:45.066802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5be604-b44c-4f51-9563-6746949bcbf7 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:45.084610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5be604-b44c-4f51-9563-6746949bcbf7 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:45.104978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:45.160239 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9de48d-d174-4f07-93b0-7575c2720ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:45.161074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9de48d-d174-4f07-93b0-7575c2720ddd admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:45.161496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9de48d-d174-4f07-93b0-7575c2720ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:45.161775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9de48d-d174-4f07-93b0-7575c2720ddd admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:45.162545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9de48d-d174-4f07-93b0-7575c2720ddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:45.164775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9de48d-d174-4f07-93b0-7575c2720ddd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:45.289011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1069/2158] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:45 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:45.293955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55f170-006e-4553-9c66-fe8cad9fe2cb admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:45.305572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55f170-006e-4553-9c66-fe8cad9fe2cb admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:45.316896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77506-4975-41e5-a552-5dae47669cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:45.316896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77506-4975-41e5-a552-5dae47669cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:45.316896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77506-4975-41e5-a552-5dae47669cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:45.316896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a77506-4975-41e5-a552-5dae47669cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:45.317335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a77506-4975-41e5-a552-5dae47669cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:45.317335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a77506-4975-41e5-a552-5dae47669cd6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:45.329321 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1090/2159] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:45.333199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9de48d-d174-4f07-93b0-7575c2720ddd admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:45.345649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9de48d-d174-4f07-93b0-7575c2720ddd admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:45.350255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d7616-3d58-4b50-8547-a19ea9e693b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:45.351165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d7616-3d58-4b50-8547-a19ea9e693b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:45.351597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d7616-3d58-4b50-8547-a19ea9e693b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:45.351886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d7616-3d58-4b50-8547-a19ea9e693b8 admin admin] PATH_INFO: `/v3/projects/93c45e598b794dfa9a497e0a3562bc03/users/e99e067f8c9745adb058e57c82d6526e/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:45.386473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d7616-3d58-4b50-8547-a19ea9e693b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c45e598b794dfa9a497e0a3562bc03, user_id=e99e067f8c9745adb058e57c82d6526e, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:45.392767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d7616-3d58-4b50-8547-a19ea9e693b8 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:45.496273 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1070/2160] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:45 2022] PUT /identity/v3/projects/93c45e598b794dfa9a497e0a3562bc03/users/e99e067f8c9745adb058e57c82d6526e/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:45.499124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77506-4975-41e5-a552-5dae47669cd6 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:45.515854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a77506-4975-41e5-a552-5dae47669cd6 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:45.542511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:45.592886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f5801-416c-463e-8a24-f365cd2bce4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:45.593773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5801-416c-463e-8a24-f365cd2bce4e admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:45.594055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5801-416c-463e-8a24-f365cd2bce4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:45.594328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f5801-416c-463e-8a24-f365cd2bce4e admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:45.595153 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f5801-416c-463e-8a24-f365cd2bce4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:45.596681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f5801-416c-463e-8a24-f365cd2bce4e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:45.609848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1091/2161] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:45.615137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d7616-3d58-4b50-8547-a19ea9e693b8 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:45.628102 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d7616-3d58-4b50-8547-a19ea9e693b8 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:45.632868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603bbe6-e5a9-45e6-92e4-a7786224a0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:45.636398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603bbe6-e5a9-45e6-92e4-a7786224a0f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:45.636398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603bbe6-e5a9-45e6-92e4-a7786224a0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:45.636398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603bbe6-e5a9-45e6-92e4-a7786224a0f7 admin admin] PATH_INFO: `/v3/projects/93c45e598b794dfa9a497e0a3562bc03/users/e99e067f8c9745adb058e57c82d6526e/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:45.636624 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603bbe6-e5a9-45e6-92e4-a7786224a0f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93c45e598b794dfa9a497e0a3562bc03, user_id=e99e067f8c9745adb058e57c82d6526e, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:45.639436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8603bbe6-e5a9-45e6-92e4-a7786224a0f7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:45.650709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1071/2162] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:45 2022] PUT /identity/v3/projects/93c45e598b794dfa9a497e0a3562bc03/users/e99e067f8c9745adb058e57c82d6526e/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:45.661784 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7acac-ddfc-441b-a6ac-8290dafae756 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:45.662480 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7acac-ddfc-441b-a6ac-8290dafae756 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:45.663442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f7acac-ddfc-441b-a6ac-8290dafae756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:45.709738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-70f7acac-ddfc-441b-a6ac-8290dafae756 None None] MFA Rules not processed for user `e99e067f8c9745adb058e57c82d6526e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:45.716507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-70f7acac-ddfc-441b-a6ac-8290dafae756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:45.809089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1092/2163] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:45.825961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603bbe6-e5a9-45e6-92e4-a7786224a0f7 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:45.827871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8603bbe6-e5a9-45e6-92e4-a7786224a0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:45.893598 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603bbe6-e5a9-45e6-92e4-a7786224a0f7 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:45.897977 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5673f-2903-430e-9330-bd83949c53e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:45.898635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5673f-2903-430e-9330-bd83949c53e3 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:45.898924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5673f-2903-430e-9330-bd83949c53e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:45.899204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5673f-2903-430e-9330-bd83949c53e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:45.921049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5673f-2903-430e-9330-bd83949c53e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:45.926832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5673f-2903-430e-9330-bd83949c53e3 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:45.944450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1072/2164] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:46.106909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7acac-ddfc-441b-a6ac-8290dafae756 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:46.108348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-70f7acac-ddfc-441b-a6ac-8290dafae756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:46.193707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f7acac-ddfc-441b-a6ac-8290dafae756 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:46.199254 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab7437-19fe-4b40-b4ac-1059338008f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:46.200268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab7437-19fe-4b40-b4ac-1059338008f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:46.200674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab7437-19fe-4b40-b4ac-1059338008f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:46.201122 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab7437-19fe-4b40-b4ac-1059338008f7 admin admin] PATH_INFO: `/v3/users/53ae3beaebb3421e9cf46068f0291f16` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:46.202420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab7437-19fe-4b40-b4ac-1059338008f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53ae3beaebb3421e9cf46068f0291f16)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:46.204358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab7437-19fe-4b40-b4ac-1059338008f7 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:46.448588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:46.451249 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-86ab7437-19fe-4b40-b4ac-1059338008f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53ae3beaebb3421e9cf46068f0291f16'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:46.451909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-86ab7437-19fe-4b40-b4ac-1059338008f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53ae3beaebb3421e9cf46068f0291f16'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:46.458762 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-86ab7437-19fe-4b40-b4ac-1059338008f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53ae3beaebb3421e9cf46068f0291f16'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:46.459386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-86ab7437-19fe-4b40-b4ac-1059338008f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53ae3beaebb3421e9cf46068f0291f16'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:46.475020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-86ab7437-19fe-4b40-b4ac-1059338008f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53ae3beaebb3421e9cf46068f0291f16'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:46.491647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5673f-2903-430e-9330-bd83949c53e3 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:46.505533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5673f-2903-430e-9330-bd83949c53e3 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:46.510849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:46.511915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:46.512193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:46.512465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] PATH_INFO: `/v3/users/5d3a20ade4eb484dbdba11f0213a52d7` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:46.513670 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d3a20ade4eb484dbdba11f0213a52d7)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:46.515729 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:46.563039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1093/2165] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:46 2022] DELETE /identity/v3/users/53ae3beaebb3421e9cf46068f0291f16 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:46.567329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf49a0e-d92e-493b-bf46-7ab8e160a18c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:46.568041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf49a0e-d92e-493b-bf46-7ab8e160a18c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:46.568041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf49a0e-d92e-493b-bf46-7ab8e160a18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:46.588400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0cf49a0e-d92e-493b-bf46-7ab8e160a18c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:46.589459 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0cf49a0e-d92e-493b-bf46-7ab8e160a18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:46.651302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1094/2166] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:46.668263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf49a0e-d92e-493b-bf46-7ab8e160a18c None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:46.669734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0cf49a0e-d92e-493b-bf46-7ab8e160a18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:46.737940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf49a0e-d92e-493b-bf46-7ab8e160a18c None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:46.738070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f4c07-2e54-49e4-93a5-c0b1f2ec5898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:46.739264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f4c07-2e54-49e4-93a5-c0b1f2ec5898 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:46.739766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f4c07-2e54-49e4-93a5-c0b1f2ec5898 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:46.740115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f4c07-2e54-49e4-93a5-c0b1f2ec5898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:46.751777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f4c07-2e54-49e4-93a5-c0b1f2ec5898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:46.753226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f4c07-2e54-49e4-93a5-c0b1f2ec5898 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:46.763188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1095/2167] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:46.801932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d3a20ade4eb484dbdba11f0213a52d7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:46.810463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d3a20ade4eb484dbdba11f0213a52d7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:46.811276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d3a20ade4eb484dbdba11f0213a52d7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:46.824525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d3a20ade4eb484dbdba11f0213a52d7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:46.825070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d3a20ade4eb484dbdba11f0213a52d7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:46.827043 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:46.934924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1073/2168] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:46 2022] DELETE /identity/v3/users/5d3a20ade4eb484dbdba11f0213a52d7 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:46.940351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5b616-e8e0-4617-95d9-7dce0adcebad None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:46.940802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5b616-e8e0-4617-95d9-7dce0adcebad None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:46.941336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5b616-e8e0-4617-95d9-7dce0adcebad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:46.960606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f1a5b616-e8e0-4617-95d9-7dce0adcebad None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:46.961840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f1a5b616-e8e0-4617-95d9-7dce0adcebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:47.046547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1096/2169] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:47.049792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:47.051230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:47.055596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5b616-e8e0-4617-95d9-7dce0adcebad None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:47.057973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f1a5b616-e8e0-4617-95d9-7dce0adcebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:47.065308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:47.068372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01355953-8dc5-4f01-8ae9-8d9e3f45c12d admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:47.076629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea970a-37d3-4e7d-8a5d-da7386111283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:47.078007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea970a-37d3-4e7d-8a5d-da7386111283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:47.078367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea970a-37d3-4e7d-8a5d-da7386111283 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:47.078769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea970a-37d3-4e7d-8a5d-da7386111283 admin admin] PATH_INFO: `/v3/users/96a6161a7cc34e41bf8618a8e5a89857` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:47.080257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea970a-37d3-4e7d-8a5d-da7386111283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a6161a7cc34e41bf8618a8e5a89857)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:47.082682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea970a-37d3-4e7d-8a5d-da7386111283 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:47.134751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5b616-e8e0-4617-95d9-7dce0adcebad None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:47.141028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cdc8b-90f4-464e-88ce-1f808157d73c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:47.141889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cdc8b-90f4-464e-88ce-1f808157d73c service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:47.142230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cdc8b-90f4-464e-88ce-1f808157d73c service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:47.142667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036cdc8b-90f4-464e-88ce-1f808157d73c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:47.155377 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cdc8b-90f4-464e-88ce-1f808157d73c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:47.156837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036cdc8b-90f4-464e-88ce-1f808157d73c service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:47.169208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1097/2170] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:47.290514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cdc8b-90f4-464e-88ce-1f808157d73c service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:47.292023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-036cdc8b-90f4-464e-88ce-1f808157d73c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:47.306444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036cdc8b-90f4-464e-88ce-1f808157d73c service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:47.312154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:47.313017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:47.313369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:47.313632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] PATH_INFO: `/v3/projects/2c53f45e2ea14ae9b8c82aa3fa271bc1` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:47.315096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c53f45e2ea14ae9b8c82aa3fa271bc1)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:47.317056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:47.348192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8cea970a-37d3-4e7d-8a5d-da7386111283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a6161a7cc34e41bf8618a8e5a89857'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.356920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8cea970a-37d3-4e7d-8a5d-da7386111283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a6161a7cc34e41bf8618a8e5a89857'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.357638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8cea970a-37d3-4e7d-8a5d-da7386111283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a6161a7cc34e41bf8618a8e5a89857'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.369506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8cea970a-37d3-4e7d-8a5d-da7386111283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a6161a7cc34e41bf8618a8e5a89857'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.370680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8cea970a-37d3-4e7d-8a5d-da7386111283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a6161a7cc34e41bf8618a8e5a89857'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.449918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1074/2171] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:47 2022] DELETE /identity/v3/users/96a6161a7cc34e41bf8618a8e5a89857 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:47.454984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7e7c-bf85-41b7-9e68-03bdbb41865e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:47.455280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7e7c-bf85-41b7-9e68-03bdbb41865e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:47.455560 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d7e7c-bf85-41b7-9e68-03bdbb41865e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:47.456055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c53f45e2ea14ae9b8c82aa3fa271bc1', 'request_id': 'req-b2bd14c8-00d8-4d34-a54e-f037a9998800'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.474068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2e8d7e7c-bf85-41b7-9e68-03bdbb41865e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:47.475357 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2e8d7e7c-bf85-41b7-9e68-03bdbb41865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:47.542416 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1075/2172] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:47.556713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7e7c-bf85-41b7-9e68-03bdbb41865e None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:47.558632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2e8d7e7c-bf85-41b7-9e68-03bdbb41865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:47.568078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c53f45e2ea14ae9b8c82aa3fa271bc1', 'request_id': 'req-b2bd14c8-00d8-4d34-a54e-f037a9998800'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.575325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c53f45e2ea14ae9b8c82aa3fa271bc1', 'request_id': 'req-b2bd14c8-00d8-4d34-a54e-f037a9998800'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.587463 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] The token cache is being invalidate because project 2c53f45e2ea14ae9b8c82aa3fa271bc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:47.587756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.588444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b2bd14c8-00d8-4d34-a54e-f037a9998800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.594534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1098/2173] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:47 2022] DELETE /identity/v3/projects/2c53f45e2ea14ae9b8c82aa3fa271bc1 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:47.594904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2d3f1-f6c0-4251-8fd1-01d9764a6bcb None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:47.595199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2d3f1-f6c0-4251-8fd1-01d9764a6bcb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:47.595479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2d3f1-f6c0-4251-8fd1-01d9764a6bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:47.625536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-eae2d3f1-f6c0-4251-8fd1-01d9764a6bcb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:47.626769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-eae2d3f1-f6c0-4251-8fd1-01d9764a6bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:47.627999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d7e7c-bf85-41b7-9e68-03bdbb41865e None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:47.631680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6822288-fa85-4a96-83d1-c37c0b808d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:47.632320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6822288-fa85-4a96-83d1-c37c0b808d41 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:47.632664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6822288-fa85-4a96-83d1-c37c0b808d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:47.632898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6822288-fa85-4a96-83d1-c37c0b808d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:47.634625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1099/2174] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:47.634767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b6822288-fa85-4a96-83d1-c37c0b808d41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:47.642854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2afb8-d80e-47e9-a207-636a4eb26be9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:47.643280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2afb8-d80e-47e9-a207-636a4eb26be9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:47.643669 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc2afb8-d80e-47e9-a207-636a4eb26be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:47.647383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6822288-fa85-4a96-83d1-c37c0b808d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:47.648883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6822288-fa85-4a96-83d1-c37c0b808d41 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:47.658881 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1076/2175] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:57:47.661882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-7bc2afb8-d80e-47e9-a207-636a4eb26be9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:47.663077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7bc2afb8-d80e-47e9-a207-636a4eb26be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:47.671788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1100/2176] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:47.676246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6822288-fa85-4a96-83d1-c37c0b808d41 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:47.686684 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6822288-fa85-4a96-83d1-c37c0b808d41 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:47.691314 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b1af7-4819-4e60-9538-aba0f05be6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:47.691961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b1af7-4819-4e60-9538-aba0f05be6ca admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:47.692273 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b1af7-4819-4e60-9538-aba0f05be6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:47.692552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b1af7-4819-4e60-9538-aba0f05be6ca admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:47.693275 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b1af7-4819-4e60-9538-aba0f05be6ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:47.693585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b1af7-4819-4e60-9538-aba0f05be6ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:47.695169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b1af7-4819-4e60-9538-aba0f05be6ca admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:47.698136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:47.710479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2afb8-d80e-47e9-a207-636a4eb26be9 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:47.711646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7bc2afb8-d80e-47e9-a207-636a4eb26be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:47.718957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1077/2177] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:47.723036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b1af7-4819-4e60-9538-aba0f05be6ca admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:47.726027 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc2afb8-d80e-47e9-a207-636a4eb26be9 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:47.732166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:47.738165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:47.738165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:47.738165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] PATH_INFO: `/v3/projects/a130a94e28254c33a6524ed44916173f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:47.738165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a130a94e28254c33a6524ed44916173f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:47.738818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b1af7-4819-4e60-9538-aba0f05be6ca admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:47.739227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:47.742736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159cd73-2bae-44a7-b13a-d1aeddc589d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:47.743411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159cd73-2bae-44a7-b13a-d1aeddc589d9 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:47.743696 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159cd73-2bae-44a7-b13a-d1aeddc589d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:47.744422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159cd73-2bae-44a7-b13a-d1aeddc589d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:47.746994 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159cd73-2bae-44a7-b13a-d1aeddc589d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:47.749778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159cd73-2bae-44a7-b13a-d1aeddc589d9 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:47.856760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a130a94e28254c33a6524ed44916173f', 'request_id': 'req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.865054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1078/2178] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:47 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:47.868938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159cd73-2bae-44a7-b13a-d1aeddc589d9 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:47.934635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a130a94e28254c33a6524ed44916173f', 'request_id': 'req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.942320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159cd73-2bae-44a7-b13a-d1aeddc589d9 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:47.944398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a130a94e28254c33a6524ed44916173f', 'request_id': 'req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.947317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e5b14-14d5-473f-a72b-f461e336b9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:47.948016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e5b14-14d5-473f-a72b-f461e336b9e7 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:47.948611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e5b14-14d5-473f-a72b-f461e336b9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:47.948611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e5b14-14d5-473f-a72b-f461e336b9e7 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:47.949356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e5b14-14d5-473f-a72b-f461e336b9e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:47.951364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e5b14-14d5-473f-a72b-f461e336b9e7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:47.953946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:47.960689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] The token cache is being invalidate because project a130a94e28254c33a6524ed44916173f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:47.960997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.962374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:47.964293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1101/2179] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:47 2022] DELETE /identity/v3/projects/a130a94e28254c33a6524ed44916173f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:47.968633 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:47.970212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:48.050587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdec1e-51f6-4bc1-aed9-36a0d2ec217d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:48.051433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1079/2180] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:47 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:48.052427 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e5b14-14d5-473f-a72b-f461e336b9e7 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:48.055171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:48.058907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d53e5b14-14d5-473f-a72b-f461e336b9e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:48.059016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:48.059016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:48.059016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] PATH_INFO: `/v3/users/e99e067f8c9745adb058e57c82d6526e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:48.059016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e99e067f8c9745adb058e57c82d6526e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:48.060960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:48.076133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e5b14-14d5-473f-a72b-f461e336b9e7 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:48.081971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1003f6b-cdf4-43ec-b168-5ecfa7483337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:48.082640 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1003f6b-cdf4-43ec-b168-5ecfa7483337 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:48.082926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1003f6b-cdf4-43ec-b168-5ecfa7483337 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:48.083203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1003f6b-cdf4-43ec-b168-5ecfa7483337 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:48.083924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1003f6b-cdf4-43ec-b168-5ecfa7483337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:48.085638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1003f6b-cdf4-43ec-b168-5ecfa7483337 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:48.096822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1080/2181] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:48.097638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1003f6b-cdf4-43ec-b168-5ecfa7483337 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:48.099835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c1003f6b-cdf4-43ec-b168-5ecfa7483337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:48.113143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1003f6b-cdf4-43ec-b168-5ecfa7483337 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:48.117737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:48.118598 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:48.118881 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:48.119153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] PATH_INFO: `/v3/users/4c35fa0c8e3b42efbe30af614b4c6db1` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:48.120331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c35fa0c8e3b42efbe30af614b4c6db1)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:48.122321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:48.318939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e99e067f8c9745adb058e57c82d6526e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.320036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e99e067f8c9745adb058e57c82d6526e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.332633 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e99e067f8c9745adb058e57c82d6526e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.333422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e99e067f8c9745adb058e57c82d6526e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.345712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c35fa0c8e3b42efbe30af614b4c6db1'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.351730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e99e067f8c9745adb058e57c82d6526e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.356862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c35fa0c8e3b42efbe30af614b4c6db1'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.357627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c35fa0c8e3b42efbe30af614b4c6db1'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.374332 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c35fa0c8e3b42efbe30af614b4c6db1'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.375066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c35fa0c8e3b42efbe30af614b4c6db1'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.436970 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1102/2182] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:47 2022] DELETE /identity/v3/users/e99e067f8c9745adb058e57c82d6526e => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:48.438096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:48.439701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:48.534718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456307a0-338f-4ebd-a44b-29be8d1fa124 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:48.537053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1081/2183] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:48 2022] DELETE /identity/v3/users/4c35fa0c8e3b42efbe30af614b4c6db1 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:48.538220 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:48.541076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:48.566736 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:48.568319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:48.599836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327035b-8843-4c5a-a8e7-3166ec481629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:48.600537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327035b-8843-4c5a-a8e7-3166ec481629 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:48.600874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327035b-8843-4c5a-a8e7-3166ec481629 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:48.601142 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327035b-8843-4c5a-a8e7-3166ec481629 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:48.601902 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327035b-8843-4c5a-a8e7-3166ec481629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:48.603518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327035b-8843-4c5a-a8e7-3166ec481629 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:48.607677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabb426-fdb2-4314-9ee9-eff20901ff7c admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:48.612272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:48.613150 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:48.613462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:48.613734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] PATH_INFO: `/v3/projects/229f5509f8bd4783a4bf6d706dd018ee` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:48.613834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1103/2184] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:48.614900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=229f5509f8bd4783a4bf6d706dd018ee)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:48.615035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71744dd6-e790-49c6-b561-646e475c2a57 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:48.615331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71744dd6-e790-49c6-b561-646e475c2a57 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:48.615614 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71744dd6-e790-49c6-b561-646e475c2a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:48.616798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:48.638222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-71744dd6-e790-49c6-b561-646e475c2a57 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:48.639454 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-71744dd6-e790-49c6-b561-646e475c2a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:48.647852 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1104/2185] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:48.648876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71744dd6-e790-49c6-b561-646e475c2a57 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:48.650153 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-71744dd6-e790-49c6-b561-646e475c2a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:48.662967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71744dd6-e790-49c6-b561-646e475c2a57 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:48.667899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afffbf9-01c2-4598-9c7d-d59dad375e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:48.668783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afffbf9-01c2-4598-9c7d-d59dad375e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:48.669068 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afffbf9-01c2-4598-9c7d-d59dad375e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:48.669532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afffbf9-01c2-4598-9c7d-d59dad375e71 admin admin] PATH_INFO: `/v3/projects/e16a74b1bc56455b8f60bfc28bc94afb/users/eed09fa6ecf14635949417f19c8a3af0/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:48.688374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afffbf9-01c2-4598-9c7d-d59dad375e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e16a74b1bc56455b8f60bfc28bc94afb, user_id=eed09fa6ecf14635949417f19c8a3af0, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:48.691338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afffbf9-01c2-4598-9c7d-d59dad375e71 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:48.764191 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1105/2186] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:48 2022] PUT /identity/v3/projects/e16a74b1bc56455b8f60bfc28bc94afb/users/eed09fa6ecf14635949417f19c8a3af0/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:48.768432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afffbf9-01c2-4598-9c7d-d59dad375e71 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:48.770292 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3afffbf9-01c2-4598-9c7d-d59dad375e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:48.774669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '229f5509f8bd4783a4bf6d706dd018ee', 'request_id': 'req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.837827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afffbf9-01c2-4598-9c7d-d59dad375e71 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:48.859607 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '229f5509f8bd4783a4bf6d706dd018ee', 'request_id': 'req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.866425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '229f5509f8bd4783a4bf6d706dd018ee', 'request_id': 'req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.879992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] The token cache is being invalidate because project 229f5509f8bd4783a4bf6d706dd018ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:48.880213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.881096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-99fa3d15-e7aa-47f5-b5d2-1c58f01f6f6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:48.882695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1082/2187] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:48 2022] DELETE /identity/v3/projects/229f5509f8bd4783a4bf6d706dd018ee => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:48.890700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4aa48-d1d5-4d3c-a7a0-47e68af2a5c1 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:48.891118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4aa48-d1d5-4d3c-a7a0-47e68af2a5c1 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:48.891656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4aa48-d1d5-4d3c-a7a0-47e68af2a5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:48.894604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4c400-67d6-406d-9060-86a01239f09f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:48.895302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4c400-67d6-406d-9060-86a01239f09f service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:48.895570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4c400-67d6-406d-9060-86a01239f09f service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:48.895848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4c400-67d6-406d-9060-86a01239f09f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:48.897724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e5c4c400-67d6-406d-9060-86a01239f09f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:48.912343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-25a4aa48-d1d5-4d3c-a7a0-47e68af2a5c1 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:48.914914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-25a4aa48-d1d5-4d3c-a7a0-47e68af2a5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:48.968569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4c400-67d6-406d-9060-86a01239f09f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:48.970451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4c400-67d6-406d-9060-86a01239f09f service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:48.976202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:48.982102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1106/2188] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:48.993515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1083/2189] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:48.993610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66976478-e667-47a4-adc3-6dd5c6441822 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:48.993610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66976478-e667-47a4-adc3-6dd5c6441822 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:48.999103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4aa48-d1d5-4d3c-a7a0-47e68af2a5c1 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:48.999193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66976478-e667-47a4-adc3-6dd5c6441822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:48.999266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-25a4aa48-d1d5-4d3c-a7a0-47e68af2a5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.008101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4aa48-d1d5-4d3c-a7a0-47e68af2a5c1 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.014631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.015567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.015926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.016395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] PATH_INFO: `/v3/projects/93c45e598b794dfa9a497e0a3562bc03` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.017525 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-66976478-e667-47a4-adc3-6dd5c6441822 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:49.017783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] RBAC: Authorizing `identity:delete_project(project_id=93c45e598b794dfa9a497e0a3562bc03)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.019174 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-66976478-e667-47a4-adc3-6dd5c6441822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.019843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:49.034686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1107/2190] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:49.034686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66976478-e667-47a4-adc3-6dd5c6441822 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:49.036031 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-66976478-e667-47a4-adc3-6dd5c6441822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.048070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66976478-e667-47a4-adc3-6dd5c6441822 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.051912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dc7ba-9356-4e87-8529-538e370fc449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.052582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dc7ba-9356-4e87-8529-538e370fc449 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.052991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dc7ba-9356-4e87-8529-538e370fc449 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.053401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dc7ba-9356-4e87-8529-538e370fc449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.063428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dc7ba-9356-4e87-8529-538e370fc449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.064938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dc7ba-9356-4e87-8529-538e370fc449 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:49.075350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1108/2191] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:49.076515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c90bf-5065-4ba4-9f45-ed206134478f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.076817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c90bf-5065-4ba4-9f45-ed206134478f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.077595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c90bf-5065-4ba4-9f45-ed206134478f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.095805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f34c90bf-5065-4ba4-9f45-ed206134478f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:49.097695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f34c90bf-5065-4ba4-9f45-ed206134478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.111806 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1109/2192] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:49.113621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c90bf-5065-4ba4-9f45-ed206134478f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:49.122864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c90bf-5065-4ba4-9f45-ed206134478f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.127572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57722f3-73d3-4e12-9d4c-a3b876cf54d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.128216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57722f3-73d3-4e12-9d4c-a3b876cf54d1 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.128498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57722f3-73d3-4e12-9d4c-a3b876cf54d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.128773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57722f3-73d3-4e12-9d4c-a3b876cf54d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.129616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57722f3-73d3-4e12-9d4c-a3b876cf54d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:49.129924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57722f3-73d3-4e12-9d4c-a3b876cf54d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.131568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57722f3-73d3-4e12-9d4c-a3b876cf54d1 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:49.143957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1110/2193] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:57:49.148137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57722f3-73d3-4e12-9d4c-a3b876cf54d1 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:49.158312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57722f3-73d3-4e12-9d4c-a3b876cf54d1 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.163053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222d5f9-a296-44d0-9c38-3dbd9726bbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.163729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222d5f9-a296-44d0-9c38-3dbd9726bbca admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.164016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222d5f9-a296-44d0-9c38-3dbd9726bbca admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.164293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222d5f9-a296-44d0-9c38-3dbd9726bbca admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.169514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222d5f9-a296-44d0-9c38-3dbd9726bbca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.169514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222d5f9-a296-44d0-9c38-3dbd9726bbca admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:49.181038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:49.182678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93c45e598b794dfa9a497e0a3562bc03', 'request_id': 'req-790230fa-5618-4c34-bf20-704b24fec61f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:49.272230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93c45e598b794dfa9a497e0a3562bc03', 'request_id': 'req-790230fa-5618-4c34-bf20-704b24fec61f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:49.283693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93c45e598b794dfa9a497e0a3562bc03', 'request_id': 'req-790230fa-5618-4c34-bf20-704b24fec61f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:49.288991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1111/2194] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:49 2022] POST /identity/v3/projects => generated 357 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:49.289954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222d5f9-a296-44d0-9c38-3dbd9726bbca admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:49.298718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] The token cache is being invalidate because project 93c45e598b794dfa9a497e0a3562bc03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:49.299014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:49.303880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-790230fa-5618-4c34-bf20-704b24fec61f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:49.303880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1084/2195] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:48 2022] DELETE /identity/v3/projects/93c45e598b794dfa9a497e0a3562bc03 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:49.309465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865307fc-b073-4973-850e-d1ae30f7a73a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.309786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865307fc-b073-4973-850e-d1ae30f7a73a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.310076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865307fc-b073-4973-850e-d1ae30f7a73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.356311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-865307fc-b073-4973-850e-d1ae30f7a73a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:49.357673 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-865307fc-b073-4973-850e-d1ae30f7a73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.393040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222d5f9-a296-44d0-9c38-3dbd9726bbca admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.400242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96383c60-0a71-486a-8b21-fa5ac93bf5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.401064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96383c60-0a71-486a-8b21-fa5ac93bf5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.402681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96383c60-0a71-486a-8b21-fa5ac93bf5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.402681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96383c60-0a71-486a-8b21-fa5ac93bf5d2 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.403033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96383c60-0a71-486a-8b21-fa5ac93bf5d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.405070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96383c60-0a71-486a-8b21-fa5ac93bf5d2 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:49.433158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1085/2196] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:49.442355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d7cfb-e290-4980-aea7-a88a617a8f46 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.442999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d7cfb-e290-4980-aea7-a88a617a8f46 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.443286 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d7cfb-e290-4980-aea7-a88a617a8f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.468251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f85d7cfb-e290-4980-aea7-a88a617a8f46 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:49.470020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f85d7cfb-e290-4980-aea7-a88a617a8f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.476623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1112/2197] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:49 2022] POST /identity/v3/users => generated 279 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:49.476623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96383c60-0a71-486a-8b21-fa5ac93bf5d2 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:49.483216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-96383c60-0a71-486a-8b21-fa5ac93bf5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.488810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1086/2198] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:49.489730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d7cfb-e290-4980-aea7-a88a617a8f46 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:49.491092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f85d7cfb-e290-4980-aea7-a88a617a8f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.512432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96383c60-0a71-486a-8b21-fa5ac93bf5d2 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.520525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d7cfb-e290-4980-aea7-a88a617a8f46 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.524216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.525079 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.525496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.525745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] PATH_INFO: `/v3/projects/0b2bae632b624df0a2bedfad6548e423` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.527979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2bae632b624df0a2bedfad6548e423)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.530163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:49.532577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3cb36-2954-4bfa-8153-3ac662045670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.533626 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3cb36-2954-4bfa-8153-3ac662045670 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.533911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3cb36-2954-4bfa-8153-3ac662045670 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.534193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3cb36-2954-4bfa-8153-3ac662045670 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.538641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3cb36-2954-4bfa-8153-3ac662045670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.538641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab3cb36-2954-4bfa-8153-3ac662045670 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:49.548712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1087/2199] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:49.549974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3cb36-2954-4bfa-8153-3ac662045670 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:49.562053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3cb36-2954-4bfa-8153-3ac662045670 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.566887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e6c9d-c0d2-42ba-a984-4225e4faca89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.567626 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e6c9d-c0d2-42ba-a984-4225e4faca89 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.567966 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e6c9d-c0d2-42ba-a984-4225e4faca89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.568308 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e6c9d-c0d2-42ba-a984-4225e4faca89 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.569151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e6c9d-c0d2-42ba-a984-4225e4faca89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:49.569533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e6c9d-c0d2-42ba-a984-4225e4faca89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.571243 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481e6c9d-c0d2-42ba-a984-4225e4faca89 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:49.586753 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1088/2200] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:49.588289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e6c9d-c0d2-42ba-a984-4225e4faca89 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:49.598321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e6c9d-c0d2-42ba-a984-4225e4faca89 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.603842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cfb6b-aece-445d-a0c4-20eacad36774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.604535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cfb6b-aece-445d-a0c4-20eacad36774 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.604819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cfb6b-aece-445d-a0c4-20eacad36774 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.605095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cfb6b-aece-445d-a0c4-20eacad36774 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.606107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cfb6b-aece-445d-a0c4-20eacad36774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.607721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40cfb6b-aece-445d-a0c4-20eacad36774 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:49.618315 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1089/2201] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:49.618627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cfb6b-aece-445d-a0c4-20eacad36774 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:49.635049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cfb6b-aece-445d-a0c4-20eacad36774 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.638902 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2bae632b624df0a2bedfad6548e423', 'request_id': 'req-6a691e3b-4985-458a-a715-f8f86ec05afc'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:49.688103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2bae632b624df0a2bedfad6548e423', 'request_id': 'req-6a691e3b-4985-458a-a715-f8f86ec05afc'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:49.694697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2bae632b624df0a2bedfad6548e423', 'request_id': 'req-6a691e3b-4985-458a-a715-f8f86ec05afc'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:49.704479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e7a1f-d112-452b-8c2f-63cb8a6c0e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.705403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e7a1f-d112-452b-8c2f-63cb8a6c0e15 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.705845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e7a1f-d112-452b-8c2f-63cb8a6c0e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.707686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e7a1f-d112-452b-8c2f-63cb8a6c0e15 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.708661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e7a1f-d112-452b-8c2f-63cb8a6c0e15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.710815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e7a1f-d112-452b-8c2f-63cb8a6c0e15 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:49.717533 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] The token cache is being invalidate because project 0b2bae632b624df0a2bedfad6548e423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:49.718140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:49.719654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:49.722577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1113/2202] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:49 2022] DELETE /identity/v3/projects/0b2bae632b624df0a2bedfad6548e423 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:49.723967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:49.727608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.744544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a691e3b-4985-458a-a715-f8f86ec05afc admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.751023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa49a0-a693-4c4f-bc98-d6bcb56e3380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.751023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa49a0-a693-4c4f-bc98-d6bcb56e3380 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.751023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa49a0-a693-4c4f-bc98-d6bcb56e3380 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.751023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa49a0-a693-4c4f-bc98-d6bcb56e3380 admin admin] PATH_INFO: `/v3/projects/6aeab773ab07408da584eab3d5ca83e6/users/bf126505c6b54113bc101e8f46dd586c/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.753758 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:49.767332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa49a0-a693-4c4f-bc98-d6bcb56e3380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aeab773ab07408da584eab3d5ca83e6, user_id=bf126505c6b54113bc101e8f46dd586c, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.770579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa49a0-a693-4c4f-bc98-d6bcb56e3380 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:49.796354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1090/2203] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:49 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:49.797706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5365a-60dd-4e5d-af58-f5755ee3ffbd None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.798105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5365a-60dd-4e5d-af58-f5755ee3ffbd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.798489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5365a-60dd-4e5d-af58-f5755ee3ffbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.820853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-16a5365a-60dd-4e5d-af58-f5755ee3ffbd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:49.822254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-16a5365a-60dd-4e5d-af58-f5755ee3ffbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.835300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:49.848986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1114/2204] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:49 2022] PUT /identity/v3/projects/6aeab773ab07408da584eab3d5ca83e6/users/bf126505c6b54113bc101e8f46dd586c/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:49.858564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa49a0-a693-4c4f-bc98-d6bcb56e3380 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:49.861730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-71aa49a0-a693-4c4f-bc98-d6bcb56e3380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.888257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1091/2205] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:49.896236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ae30d-1343-4b05-a83b-f18b5269aa4e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.896236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ae30d-1343-4b05-a83b-f18b5269aa4e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.896236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ae30d-1343-4b05-a83b-f18b5269aa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.918954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-978ae30d-1343-4b05-a83b-f18b5269aa4e None None] MFA Rules not processed for user `bf126505c6b54113bc101e8f46dd586c`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:49.920280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-978ae30d-1343-4b05-a83b-f18b5269aa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:49.955465 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa49a0-a693-4c4f-bc98-d6bcb56e3380 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:49.966765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9569759-da24-4eed-a72a-f98cec794ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:49.966765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9569759-da24-4eed-a72a-f98cec794ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:49.966765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9569759-da24-4eed-a72a-f98cec794ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:49.966765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9569759-da24-4eed-a72a-f98cec794ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:49.967269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9569759-da24-4eed-a72a-f98cec794ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:49.969208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9569759-da24-4eed-a72a-f98cec794ff5 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.000568 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1092/2206] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:49 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:50.002426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d60a18-1f25-4b8f-b52a-ca5d9f020ab8 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.002723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d60a18-1f25-4b8f-b52a-ca5d9f020ab8 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.003610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d60a18-1f25-4b8f-b52a-ca5d9f020ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.023763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f7d60a18-1f25-4b8f-b52a-ca5d9f020ab8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:50.030092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f7d60a18-1f25-4b8f-b52a-ca5d9f020ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:50.037525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1093/2207] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:50.038469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d60a18-1f25-4b8f-b52a-ca5d9f020ab8 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.041402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f7d60a18-1f25-4b8f-b52a-ca5d9f020ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:50.067031 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1115/2208] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:49 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:50.068150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9569759-da24-4eed-a72a-f98cec794ff5 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.080885 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9569759-da24-4eed-a72a-f98cec794ff5 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.086117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f3337-92f1-4aaf-813c-86e9bdb079e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.086854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f3337-92f1-4aaf-813c-86e9bdb079e7 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.087201 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f3337-92f1-4aaf-813c-86e9bdb079e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.087485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f3337-92f1-4aaf-813c-86e9bdb079e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.088208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f3337-92f1-4aaf-813c-86e9bdb079e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:50.088581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f3337-92f1-4aaf-813c-86e9bdb079e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.090209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f3337-92f1-4aaf-813c-86e9bdb079e7 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.103835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1116/2209] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:50.103835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f3337-92f1-4aaf-813c-86e9bdb079e7 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.104652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d60a18-1f25-4b8f-b52a-ca5d9f020ab8 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.108635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202751d-3d8b-449c-8d1d-37463e3491c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.109305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202751d-3d8b-449c-8d1d-37463e3491c2 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.109718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202751d-3d8b-449c-8d1d-37463e3491c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.109974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202751d-3d8b-449c-8d1d-37463e3491c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.114470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f3337-92f1-4aaf-813c-86e9bdb079e7 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.118545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202751d-3d8b-449c-8d1d-37463e3491c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.119070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54649af7-73ed-49dc-8116-3ce38e8d0c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.122010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54649af7-73ed-49dc-8116-3ce38e8d0c6d admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.122010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54649af7-73ed-49dc-8116-3ce38e8d0c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.122184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202751d-3d8b-449c-8d1d-37463e3491c2 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.122271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54649af7-73ed-49dc-8116-3ce38e8d0c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.122271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54649af7-73ed-49dc-8116-3ce38e8d0c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.123065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54649af7-73ed-49dc-8116-3ce38e8d0c6d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.129630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1094/2210] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:50 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:50.130418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202751d-3d8b-449c-8d1d-37463e3491c2 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.136011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1117/2211] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:50.138018 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54649af7-73ed-49dc-8116-3ce38e8d0c6d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.142267 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202751d-3d8b-449c-8d1d-37463e3491c2 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.147116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688597e-51ab-4257-8e9c-cc5268497087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.147962 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688597e-51ab-4257-8e9c-cc5268497087 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.148256 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688597e-51ab-4257-8e9c-cc5268497087 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.148684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54649af7-73ed-49dc-8116-3ce38e8d0c6d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.153643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688597e-51ab-4257-8e9c-cc5268497087 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.154575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90d3df-b19f-426e-9c1e-ba581e61988b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.155328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90d3df-b19f-426e-9c1e-ba581e61988b admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.156572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90d3df-b19f-426e-9c1e-ba581e61988b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.156572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90d3df-b19f-426e-9c1e-ba581e61988b admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.156954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90d3df-b19f-426e-9c1e-ba581e61988b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.158108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688597e-51ab-4257-8e9c-cc5268497087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.160193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688597e-51ab-4257-8e9c-cc5268497087 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.162002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90d3df-b19f-426e-9c1e-ba581e61988b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.176237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1118/2212] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:50.176237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90d3df-b19f-426e-9c1e-ba581e61988b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.186243 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90d3df-b19f-426e-9c1e-ba581e61988b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.191331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d617b-171c-4a47-b971-c47ad21572f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.192197 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d617b-171c-4a47-b971-c47ad21572f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.192481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d617b-171c-4a47-b971-c47ad21572f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.192765 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d617b-171c-4a47-b971-c47ad21572f4 admin admin] PATH_INFO: `/v3/users/bf126505c6b54113bc101e8f46dd586c` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.193972 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d617b-171c-4a47-b971-c47ad21572f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf126505c6b54113bc101e8f46dd586c)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.195989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d617b-171c-4a47-b971-c47ad21572f4 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.273555 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1095/2213] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:50 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:50.274370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688597e-51ab-4257-8e9c-cc5268497087 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.339074 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688597e-51ab-4257-8e9c-cc5268497087 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.343792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a49b7-0c8b-4779-8d9d-274b03aeddd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.344679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a49b7-0c8b-4779-8d9d-274b03aeddd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.344961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a49b7-0c8b-4779-8d9d-274b03aeddd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.345260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a49b7-0c8b-4779-8d9d-274b03aeddd6 admin admin] PATH_INFO: `/v3/projects/22d72098e908411a921a3e8446224518/users/746df209c8e04d5281e398ec8d7a1112/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.363041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a49b7-0c8b-4779-8d9d-274b03aeddd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d72098e908411a921a3e8446224518, user_id=746df209c8e04d5281e398ec8d7a1112, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.365793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a49b7-0c8b-4779-8d9d-274b03aeddd6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.392688 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f60d617b-171c-4a47-b971-c47ad21572f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf126505c6b54113bc101e8f46dd586c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:50.393788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f60d617b-171c-4a47-b971-c47ad21572f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf126505c6b54113bc101e8f46dd586c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:50.402373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f60d617b-171c-4a47-b971-c47ad21572f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf126505c6b54113bc101e8f46dd586c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:50.403121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f60d617b-171c-4a47-b971-c47ad21572f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf126505c6b54113bc101e8f46dd586c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:50.416818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f60d617b-171c-4a47-b971-c47ad21572f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf126505c6b54113bc101e8f46dd586c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:50.469060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1096/2214] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:50 2022] PUT /identity/v3/projects/22d72098e908411a921a3e8446224518/users/746df209c8e04d5281e398ec8d7a1112/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:50.470185 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a49b7-0c8b-4779-8d9d-274b03aeddd6 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.472127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-bb5a49b7-0c8b-4779-8d9d-274b03aeddd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:50.496010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:50.505512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1119/2215] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:50 2022] DELETE /identity/v3/users/bf126505c6b54113bc101e8f46dd586c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:50.511052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d7dd9-b159-438f-adcd-e37365e7a9a4 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.511352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d7dd9-b159-438f-adcd-e37365e7a9a4 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.512143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d7dd9-b159-438f-adcd-e37365e7a9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.537962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:50.546279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-987d7dd9-b159-438f-adcd-e37365e7a9a4 None None] MFA Rules not processed for user `746df209c8e04d5281e398ec8d7a1112`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:50.547509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-987d7dd9-b159-438f-adcd-e37365e7a9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:50.580841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a49b7-0c8b-4779-8d9d-274b03aeddd6 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.635249 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1120/2216] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:50.637311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03cd0e-15f3-4fe3-addd-597f5f10cd2b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.637676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03cd0e-15f3-4fe3-addd-597f5f10cd2b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.638215 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03cd0e-15f3-4fe3-addd-597f5f10cd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.662915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72306e23-62b5-4aa2-a8a7-fba755f70c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.663707 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72306e23-62b5-4aa2-a8a7-fba755f70c7a admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.666279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72306e23-62b5-4aa2-a8a7-fba755f70c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.666279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72306e23-62b5-4aa2-a8a7-fba755f70c7a admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.666279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72306e23-62b5-4aa2-a8a7-fba755f70c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.678106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72306e23-62b5-4aa2-a8a7-fba755f70c7a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.678281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-8a03cd0e-15f3-4fe3-addd-597f5f10cd2b None None] MFA Rules not processed for user `eed09fa6ecf14635949417f19c8a3af0`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:50.680043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8a03cd0e-15f3-4fe3-addd-597f5f10cd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:50.740282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1097/2217] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:50 2022] POST /identity/v3/users => generated 283 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:50.740522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36c949-5e05-4cd0-b29f-db713c8d7522 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.740775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36c949-5e05-4cd0-b29f-db713c8d7522 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.741129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36c949-5e05-4cd0-b29f-db713c8d7522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.759957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1121/2218] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:50.761434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ac36c949-5e05-4cd0-b29f-db713c8d7522 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:50.762212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03cd0e-15f3-4fe3-addd-597f5f10cd2b None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.766556 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ac36c949-5e05-4cd0-b29f-db713c8d7522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:50.766661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8a03cd0e-15f3-4fe3-addd-597f5f10cd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:50.777667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1098/2219] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:50.778637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36c949-5e05-4cd0-b29f-db713c8d7522 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.789301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36c949-5e05-4cd0-b29f-db713c8d7522 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.795331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2669056-041d-41d7-a443-cc91a45c28cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.799437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2669056-041d-41d7-a443-cc91a45c28cc admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.799437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2669056-041d-41d7-a443-cc91a45c28cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.799437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2669056-041d-41d7-a443-cc91a45c28cc admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.799437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2669056-041d-41d7-a443-cc91a45c28cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.799437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2669056-041d-41d7-a443-cc91a45c28cc admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.813280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1099/2220] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:50.817502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2669056-041d-41d7-a443-cc91a45c28cc admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.819945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a2669056-041d-41d7-a443-cc91a45c28cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:50.832004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03cd0e-15f3-4fe3-addd-597f5f10cd2b None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.835820 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e0b7f-4bfe-406e-9532-cbe9be576056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.836554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e0b7f-4bfe-406e-9532-cbe9be576056 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.836918 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e0b7f-4bfe-406e-9532-cbe9be576056 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.837293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9e0b7f-4bfe-406e-9532-cbe9be576056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.846418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e0b7f-4bfe-406e-9532-cbe9be576056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.847949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9e0b7f-4bfe-406e-9532-cbe9be576056 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.852206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:50.858926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1122/2221] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:50.859826 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e0b7f-4bfe-406e-9532-cbe9be576056 service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.861186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ee9e0b7f-4bfe-406e-9532-cbe9be576056 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:50.882297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2669056-041d-41d7-a443-cc91a45c28cc admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.887148 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d715fc4-6d0d-473d-affe-60b26efc5000 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.888249 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d715fc4-6d0d-473d-affe-60b26efc5000 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.889754 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d715fc4-6d0d-473d-affe-60b26efc5000 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.889754 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d715fc4-6d0d-473d-affe-60b26efc5000 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.900254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d715fc4-6d0d-473d-affe-60b26efc5000 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.901944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d715fc4-6d0d-473d-affe-60b26efc5000 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.917857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1100/2222] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:50.919538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d715fc4-6d0d-473d-affe-60b26efc5000 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.919974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:50.933605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d715fc4-6d0d-473d-affe-60b26efc5000 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.938396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea867ab-9efd-411b-8e23-24032a44d510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.939192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea867ab-9efd-411b-8e23-24032a44d510 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.939479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea867ab-9efd-411b-8e23-24032a44d510 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.939753 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea867ab-9efd-411b-8e23-24032a44d510 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.940471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea867ab-9efd-411b-8e23-24032a44d510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.942251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea867ab-9efd-411b-8e23-24032a44d510 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.946472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9e0b7f-4bfe-406e-9532-cbe9be576056 service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.951435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a0ed9-0795-4187-8abe-4325c30a0ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.952157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a0ed9-0795-4187-8abe-4325c30a0ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.952516 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a0ed9-0795-4187-8abe-4325c30a0ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.952863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1101/2223] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:50.953202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a0ed9-0795-4187-8abe-4325c30a0ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.956919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea867ab-9efd-411b-8e23-24032a44d510 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.958597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0ea867ab-9efd-411b-8e23-24032a44d510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:50.965008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a0ed9-0795-4187-8abe-4325c30a0ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.966727 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a0ed9-0795-4187-8abe-4325c30a0ae5 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.971772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea867ab-9efd-411b-8e23-24032a44d510 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:50.976943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3688bb1-b507-45e4-8602-6a280a555b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:50.977922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3688bb1-b507-45e4-8602-6a280a555b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:50.978211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3688bb1-b507-45e4-8602-6a280a555b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:50.978547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1123/2224] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:50.983060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3688bb1-b507-45e4-8602-6a280a555b80 admin admin] PATH_INFO: `/v3/users/746df209c8e04d5281e398ec8d7a1112` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:50.983060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3688bb1-b507-45e4-8602-6a280a555b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=746df209c8e04d5281e398ec8d7a1112)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:50.983060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3688bb1-b507-45e4-8602-6a280a555b80 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:50.987570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a0ed9-0795-4187-8abe-4325c30a0ae5 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:50.997822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a0ed9-0795-4187-8abe-4325c30a0ae5 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:51.002862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6aca83-bda4-4332-bb3d-a42c46888b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:51.003739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aca83-bda4-4332-bb3d-a42c46888b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:51.004027 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aca83-bda4-4332-bb3d-a42c46888b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:51.004309 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6aca83-bda4-4332-bb3d-a42c46888b04 admin admin] PATH_INFO: `/v3/projects/f77b565d29b44bedab9c0ac1b8cc4c63/users/40084b453941407489dc6ae3f3f01243/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:51.022999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6aca83-bda4-4332-bb3d-a42c46888b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77b565d29b44bedab9c0ac1b8cc4c63, user_id=40084b453941407489dc6ae3f3f01243, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:51.026171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6aca83-bda4-4332-bb3d-a42c46888b04 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:51.107118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1124/2225] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:50 2022] PUT /identity/v3/projects/f77b565d29b44bedab9c0ac1b8cc4c63/users/40084b453941407489dc6ae3f3f01243/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:51.121045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7c010-0ccf-4367-8db8-b960e5bf8948 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:51.121485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7c010-0ccf-4367-8db8-b960e5bf8948 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:51.121788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7c010-0ccf-4367-8db8-b960e5bf8948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:51.149264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d0c7c010-0ccf-4367-8db8-b960e5bf8948 None None] MFA Rules not processed for user `40084b453941407489dc6ae3f3f01243`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:51.150548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d0c7c010-0ccf-4367-8db8-b960e5bf8948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:51.165735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-e3688bb1-b507-45e4-8602-6a280a555b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746df209c8e04d5281e398ec8d7a1112'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:51.173232 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-e3688bb1-b507-45e4-8602-6a280a555b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746df209c8e04d5281e398ec8d7a1112'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:51.174025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-e3688bb1-b507-45e4-8602-6a280a555b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746df209c8e04d5281e398ec8d7a1112'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:51.188053 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-e3688bb1-b507-45e4-8602-6a280a555b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746df209c8e04d5281e398ec8d7a1112'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:51.190386 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-e3688bb1-b507-45e4-8602-6a280a555b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746df209c8e04d5281e398ec8d7a1112'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:51.226237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1125/2226] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:51.238523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7c010-0ccf-4367-8db8-b960e5bf8948 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:51.239932 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d0c7c010-0ccf-4367-8db8-b960e5bf8948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:51.272734 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1102/2227] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:57:50 2022] DELETE /identity/v3/users/746df209c8e04d5281e398ec8d7a1112 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:51.277792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78ae13-f8ce-42ca-9c71-ed3251665df8 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:51.278082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78ae13-f8ce-42ca-9c71-ed3251665df8 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:51.278361 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78ae13-f8ce-42ca-9c71-ed3251665df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:51.311895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ca78ae13-f8ce-42ca-9c71-ed3251665df8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:51.313163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ca78ae13-f8ce-42ca-9c71-ed3251665df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:51.315341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7c010-0ccf-4367-8db8-b960e5bf8948 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:51.375322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825e5ca-5f1f-420a-803a-4d2c43c768b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:51.375974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825e5ca-5f1f-420a-803a-4d2c43c768b7 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:51.376264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825e5ca-5f1f-420a-803a-4d2c43c768b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:51.376547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825e5ca-5f1f-420a-803a-4d2c43c768b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:51.387901 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:51.394287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1103/2228] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:51.402250 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78ae13-f8ce-42ca-9c71-ed3251665df8 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:51.407088 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ca78ae13-f8ce-42ca-9c71-ed3251665df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:51.438523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825e5ca-5f1f-420a-803a-4d2c43c768b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:51.440003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825e5ca-5f1f-420a-803a-4d2c43c768b7 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:51.445553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:51.449637 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1126/2229] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:51 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:57:51.461718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825e5ca-5f1f-420a-803a-4d2c43c768b7 service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:51.463585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0825e5ca-5f1f-420a-803a-4d2c43c768b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:51.483636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825e5ca-5f1f-420a-803a-4d2c43c768b7 service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:51.488983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:51.490029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:51.490404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:51.490687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] PATH_INFO: `/v3/projects/6aeab773ab07408da584eab3d5ca83e6` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:51.490917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78ae13-f8ce-42ca-9c71-ed3251665df8 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:51.492037 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aeab773ab07408da584eab3d5ca83e6)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:51.494204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:51.500700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06387c-9add-47ed-941b-6a25d814bb55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:51.502321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06387c-9add-47ed-941b-6a25d814bb55 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:51.503094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06387c-9add-47ed-941b-6a25d814bb55 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:51.503621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06387c-9add-47ed-941b-6a25d814bb55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:51.521386 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06387c-9add-47ed-941b-6a25d814bb55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:51.524181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06387c-9add-47ed-941b-6a25d814bb55 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:51.538829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1104/2230] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:57:51.633929 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aeab773ab07408da584eab3d5ca83e6', 'request_id': 'req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:51.713562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aeab773ab07408da584eab3d5ca83e6', 'request_id': 'req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:51.720118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aeab773ab07408da584eab3d5ca83e6', 'request_id': 'req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:51.734720 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] The token cache is being invalidate because project 6aeab773ab07408da584eab3d5ca83e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:51.735014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:51.735664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1bdc4a2f-3ed2-47cf-a158-d263c3f8fb03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:51.737609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1127/2231] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:51 2022] DELETE /identity/v3/projects/6aeab773ab07408da584eab3d5ca83e6 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:51.742309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4912c0-cba9-4ce3-b80f-b3e434b2df4e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:51.747510 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4912c0-cba9-4ce3-b80f-b3e434b2df4e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:51.747510 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4912c0-cba9-4ce3-b80f-b3e434b2df4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:51.764612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ca4912c0-cba9-4ce3-b80f-b3e434b2df4e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:51.766118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ca4912c0-cba9-4ce3-b80f-b3e434b2df4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:51.837555 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1105/2232] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:51.845802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a935a8-9520-4fd1-a6ad-660fa828815f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:51.846097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a935a8-9520-4fd1-a6ad-660fa828815f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:51.846378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a935a8-9520-4fd1-a6ad-660fa828815f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:51.866486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-85a935a8-9520-4fd1-a6ad-660fa828815f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:51.868216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-85a935a8-9520-4fd1-a6ad-660fa828815f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:51.878766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1128/2233] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:51.883161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4912c0-cba9-4ce3-b80f-b3e434b2df4e None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:51.893413 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4912c0-cba9-4ce3-b80f-b3e434b2df4e None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:51.902467 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253056e-7f47-4dee-8241-0be5fa6a9361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:51.903134 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253056e-7f47-4dee-8241-0be5fa6a9361 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:51.903403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253056e-7f47-4dee-8241-0be5fa6a9361 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:51.903676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253056e-7f47-4dee-8241-0be5fa6a9361 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:51.904390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253056e-7f47-4dee-8241-0be5fa6a9361 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:51.904697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253056e-7f47-4dee-8241-0be5fa6a9361 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:51.906438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b253056e-7f47-4dee-8241-0be5fa6a9361 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:51.922304 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1106/2234] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:57:51.927684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a935a8-9520-4fd1-a6ad-660fa828815f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:51.936925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a935a8-9520-4fd1-a6ad-660fa828815f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:51.941553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd327738-be2d-40b3-a828-b331e694c57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:51.942198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd327738-be2d-40b3-a828-b331e694c57f admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:51.942485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd327738-be2d-40b3-a828-b331e694c57f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:51.942759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd327738-be2d-40b3-a828-b331e694c57f admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:51.943448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd327738-be2d-40b3-a828-b331e694c57f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:51.945485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd327738-be2d-40b3-a828-b331e694c57f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.035024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253056e-7f47-4dee-8241-0be5fa6a9361 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.037398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b253056e-7f47-4dee-8241-0be5fa6a9361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:52.051207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253056e-7f47-4dee-8241-0be5fa6a9361 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.056161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.057093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.057749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] PATH_INFO: `/v3/projects/22d72098e908411a921a3e8446224518` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.058973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d72098e908411a921a3e8446224518)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.060915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.076184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1129/2235] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:51 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:57:52.080566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd327738-be2d-40b3-a828-b331e694c57f admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.181615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d72098e908411a921a3e8446224518', 'request_id': 'req-a042915a-cc35-43ca-bf97-b5c424ab4d41'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:52.196111 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd327738-be2d-40b3-a828-b331e694c57f admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.208258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:52.255557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b6f17-dee3-4a4f-9bec-2714d5c42584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.256237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b6f17-dee3-4a4f-9bec-2714d5c42584 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.256521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b6f17-dee3-4a4f-9bec-2714d5c42584 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.256799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b6f17-dee3-4a4f-9bec-2714d5c42584 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.257527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b6f17-dee3-4a4f-9bec-2714d5c42584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.259509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b6f17-dee3-4a4f-9bec-2714d5c42584 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.265978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d72098e908411a921a3e8446224518', 'request_id': 'req-a042915a-cc35-43ca-bf97-b5c424ab4d41'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:52.280659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d72098e908411a921a3e8446224518', 'request_id': 'req-a042915a-cc35-43ca-bf97-b5c424ab4d41'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:52.295152 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] The token cache is being invalidate because project 22d72098e908411a921a3e8446224518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:57:52.295536 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:52.296279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a042915a-cc35-43ca-bf97-b5c424ab4d41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:57:52.298495 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1107/2236] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:57:52 2022] DELETE /identity/v3/projects/22d72098e908411a921a3e8446224518 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:52.303479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13608003-10a4-4f92-9c99-e1ef44c7171e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.303913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13608003-10a4-4f92-9c99-e1ef44c7171e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.304272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13608003-10a4-4f92-9c99-e1ef44c7171e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.321936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-13608003-10a4-4f92-9c99-e1ef44c7171e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:52.323728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-13608003-10a4-4f92-9c99-e1ef44c7171e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:52.331757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1108/2237] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:52.339962 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b781f-c93c-42e0-90b4-fa02b06bec14 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.340251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b781f-c93c-42e0-90b4-fa02b06bec14 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.340530 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b781f-c93c-42e0-90b4-fa02b06bec14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.348110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1130/2238] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:52 2022] POST /identity/v3/users => generated 288 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:52.351887 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b6f17-dee3-4a4f-9bec-2714d5c42584 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.353607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-596b6f17-dee3-4a4f-9bec-2714d5c42584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:52.363245 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-217b781f-c93c-42e0-90b4-fa02b06bec14 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:52.364962 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-217b781f-c93c-42e0-90b4-fa02b06bec14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:52.371922 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b6f17-dee3-4a4f-9bec-2714d5c42584 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.376406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1109/2239] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:52.380018 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841691f8-6de7-47eb-bbf5-66b98819b681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.380737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841691f8-6de7-47eb-bbf5-66b98819b681 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.380910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b781f-c93c-42e0-90b4-fa02b06bec14 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.381031 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841691f8-6de7-47eb-bbf5-66b98819b681 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.381373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841691f8-6de7-47eb-bbf5-66b98819b681 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.382099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841691f8-6de7-47eb-bbf5-66b98819b681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.383701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841691f8-6de7-47eb-bbf5-66b98819b681 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.390956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b781f-c93c-42e0-90b4-fa02b06bec14 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.397454 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1131/2240] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:52.397590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69573150-90dc-4196-84a6-1fdb48753233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.397590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69573150-90dc-4196-84a6-1fdb48753233 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.397590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69573150-90dc-4196-84a6-1fdb48753233 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.397590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69573150-90dc-4196-84a6-1fdb48753233 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.398036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69573150-90dc-4196-84a6-1fdb48753233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:57:52.398286 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69573150-90dc-4196-84a6-1fdb48753233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.398636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841691f8-6de7-47eb-bbf5-66b98819b681 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.399869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69573150-90dc-4196-84a6-1fdb48753233 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.409058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841691f8-6de7-47eb-bbf5-66b98819b681 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.414121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b1de9-d484-4f1e-81f0-6a9fd677837b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.414616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1110/2241] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:57:52.418248 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b1de9-d484-4f1e-81f0-6a9fd677837b admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.418663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b1de9-d484-4f1e-81f0-6a9fd677837b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.418973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69573150-90dc-4196-84a6-1fdb48753233 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.419398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b1de9-d484-4f1e-81f0-6a9fd677837b admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.420283 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b1de9-d484-4f1e-81f0-6a9fd677837b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.422629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b1de9-d484-4f1e-81f0-6a9fd677837b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.428890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69573150-90dc-4196-84a6-1fdb48753233 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.433794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1132/2242] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:52.434149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a9315-c823-45a7-901e-9b346d9b4dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.434840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a9315-c823-45a7-901e-9b346d9b4dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.435126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a9315-c823-45a7-901e-9b346d9b4dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.435411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a9315-c823-45a7-901e-9b346d9b4dc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.436139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a9315-c823-45a7-901e-9b346d9b4dc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.437713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b1de9-d484-4f1e-81f0-6a9fd677837b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.438098 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18a9315-c823-45a7-901e-9b346d9b4dc9 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.447069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b1de9-d484-4f1e-81f0-6a9fd677837b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.452499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd282a6-342a-4d6d-8861-02e891538d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.453389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd282a6-342a-4d6d-8861-02e891538d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.453676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd282a6-342a-4d6d-8861-02e891538d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.453947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd282a6-342a-4d6d-8861-02e891538d1d admin admin] PATH_INFO: `/v3/projects/2d8ec4168af342408ecb574617819cdd/users/5fb42e8a087d4d7983f8733c7eb337c7/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.454329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:52.468262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd282a6-342a-4d6d-8861-02e891538d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d8ec4168af342408ecb574617819cdd, user_id=5fb42e8a087d4d7983f8733c7eb337c7, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.470966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd282a6-342a-4d6d-8861-02e891538d1d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.514790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1111/2243] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:52 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:52.518157 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a9315-c823-45a7-901e-9b346d9b4dc9 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.530162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1133/2244] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:52 2022] PUT /identity/v3/projects/2d8ec4168af342408ecb574617819cdd/users/5fb42e8a087d4d7983f8733c7eb337c7/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:57:52.540151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddc4ee-82b5-491b-82b0-4bd1a6dcf1d7 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.540441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddc4ee-82b5-491b-82b0-4bd1a6dcf1d7 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.540719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddc4ee-82b5-491b-82b0-4bd1a6dcf1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.566867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-acddc4ee-82b5-491b-82b0-4bd1a6dcf1d7 None None] MFA Rules not processed for user `5fb42e8a087d4d7983f8733c7eb337c7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:52.568116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-acddc4ee-82b5-491b-82b0-4bd1a6dcf1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:52.580517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18a9315-c823-45a7-901e-9b346d9b4dc9 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.637929 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1134/2245] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:52.642050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7826c1-b7c0-452f-a2dd-fec65bbd34fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.642988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7826c1-b7c0-452f-a2dd-fec65bbd34fa admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.643284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7826c1-b7c0-452f-a2dd-fec65bbd34fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.643598 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7826c1-b7c0-452f-a2dd-fec65bbd34fa admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.644247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7826c1-b7c0-452f-a2dd-fec65bbd34fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.646478 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7826c1-b7c0-452f-a2dd-fec65bbd34fa admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.649429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddc4ee-82b5-491b-82b0-4bd1a6dcf1d7 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.650837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-acddc4ee-82b5-491b-82b0-4bd1a6dcf1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:52.708378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:52.714196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddc4ee-82b5-491b-82b0-4bd1a6dcf1d7 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.717997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac548083-840a-47e4-b5a2-8ecf45579b77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.718753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac548083-840a-47e4-b5a2-8ecf45579b77 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.719040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac548083-840a-47e4-b5a2-8ecf45579b77 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.719395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac548083-840a-47e4-b5a2-8ecf45579b77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.728054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac548083-840a-47e4-b5a2-8ecf45579b77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.730087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac548083-840a-47e4-b5a2-8ecf45579b77 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.748463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1112/2246] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:57:52 2022] POST /identity/v3/users => generated 290 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:57:52.748569 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1135/2247] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:57:52.748659 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7826c1-b7c0-452f-a2dd-fec65bbd34fa admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.750422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7826c1-b7c0-452f-a2dd-fec65bbd34fa admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.755118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80b1a-cf3c-42c8-8013-3f7f75a29905 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.755583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80b1a-cf3c-42c8-8013-3f7f75a29905 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.755816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451fb6f-0fd8-458e-a729-8150f23d2a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.756070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80b1a-cf3c-42c8-8013-3f7f75a29905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.756489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451fb6f-0fd8-458e-a729-8150f23d2a57 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.756904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451fb6f-0fd8-458e-a729-8150f23d2a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.757117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451fb6f-0fd8-458e-a729-8150f23d2a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.757770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451fb6f-0fd8-458e-a729-8150f23d2a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.759663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451fb6f-0fd8-458e-a729-8150f23d2a57 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.771696 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1113/2248] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:57:52.776565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-c3e80b1a-cf3c-42c8-8013-3f7f75a29905 None None] MFA Rules not processed for user `eed09fa6ecf14635949417f19c8a3af0`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:52.777914 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c3e80b1a-cf3c-42c8-8013-3f7f75a29905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:52.780815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451fb6f-0fd8-458e-a729-8150f23d2a57 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.791420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451fb6f-0fd8-458e-a729-8150f23d2a57 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.797101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d03bd1-7f5c-49c0-a13b-9ea3e3f17474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.797793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d03bd1-7f5c-49c0-a13b-9ea3e3f17474 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.798083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d03bd1-7f5c-49c0-a13b-9ea3e3f17474 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.798416 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d03bd1-7f5c-49c0-a13b-9ea3e3f17474 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.799188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d03bd1-7f5c-49c0-a13b-9ea3e3f17474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.800932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d03bd1-7f5c-49c0-a13b-9ea3e3f17474 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.813142 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1114/2249] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 18:57:52.817567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d03bd1-7f5c-49c0-a13b-9ea3e3f17474 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:52.827545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d03bd1-7f5c-49c0-a13b-9ea3e3f17474 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:52.838000 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb907a-2e7d-4fed-bcee-80871091cddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:52.839837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb907a-2e7d-4fed-bcee-80871091cddb admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.840125 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb907a-2e7d-4fed-bcee-80871091cddb admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.840399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb907a-2e7d-4fed-bcee-80871091cddb admin admin] PATH_INFO: `/v3/projects/fc32e3179c534d83b3d4108a38fb9d19/users/1c0e7e9f9d0f409a8820d8bd8dd239cf/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.846298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1136/2250] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:57:52.858675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb907a-2e7d-4fed-bcee-80871091cddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc32e3179c534d83b3d4108a38fb9d19, user_id=1c0e7e9f9d0f409a8820d8bd8dd239cf, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:52.861350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb907a-2e7d-4fed-bcee-80871091cddb admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:52.963559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1115/2251] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:57:52 2022] PUT /identity/v3/projects/fc32e3179c534d83b3d4108a38fb9d19/users/1c0e7e9f9d0f409a8820d8bd8dd239cf/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:57:52.969421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4b32-f376-4e13-ba75-4db30404d399 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:52.969720 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4b32-f376-4e13-ba75-4db30404d399 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:52.970003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e4b32-f376-4e13-ba75-4db30404d399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:52.989493 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9f5e4b32-f376-4e13-ba75-4db30404d399 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:52.990846 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9f5e4b32-f376-4e13-ba75-4db30404d399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:53.052936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1137/2252] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:53.079526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb907a-2e7d-4fed-bcee-80871091cddb admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:53.081746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-21eb907a-2e7d-4fed-bcee-80871091cddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:53.134229 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:57:53.140282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb907a-2e7d-4fed-bcee-80871091cddb admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:53.150573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08261aa5-5bc3-4c03-9dd7-344d418c4493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:53.152064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08261aa5-5bc3-4c03-9dd7-344d418c4493 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:53.152356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08261aa5-5bc3-4c03-9dd7-344d418c4493 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:53.153327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08261aa5-5bc3-4c03-9dd7-344d418c4493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:53.164602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08261aa5-5bc3-4c03-9dd7-344d418c4493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:53.166811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08261aa5-5bc3-4c03-9dd7-344d418c4493 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:57:53.191231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1116/2253] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:57:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:57:59.698223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c48a1-faec-4386-b52f-0298e70cd28a None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:59.699375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c48a1-faec-4386-b52f-0298e70cd28a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:59.699984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c48a1-faec-4386-b52f-0298e70cd28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:59.790344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-570c48a1-faec-4386-b52f-0298e70cd28a None None] MFA Rules not processed for user `1c0e7e9f9d0f409a8820d8bd8dd239cf`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:57:59.790344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-570c48a1-faec-4386-b52f-0298e70cd28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:57:59.908858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1138/2254] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:57:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:57:59.913144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08261aa5-5bc3-4c03-9dd7-344d418c4493 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:57:59.933437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08261aa5-5bc3-4c03-9dd7-344d418c4493 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:57:59.968273 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00442157-076f-4d01-be4b-66d49351e4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:57:59.968949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00442157-076f-4d01-be4b-66d49351e4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:57:59.969269 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00442157-076f-4d01-be4b-66d49351e4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:57:59.969546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00442157-076f-4d01-be4b-66d49351e4b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:57:59.970257 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00442157-076f-4d01-be4b-66d49351e4b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:57:59.992760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00442157-076f-4d01-be4b-66d49351e4b6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:00.143860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1117/2255] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:57:59 2022] POST /identity/v3/projects => generated 377 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:00.148220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c48a1-faec-4386-b52f-0298e70cd28a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:00.160641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c48a1-faec-4386-b52f-0298e70cd28a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:00.180692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:00.217874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8083050-8a61-4949-997d-1d370869181e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:00.218607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083050-8a61-4949-997d-1d370869181e admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:00.218854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083050-8a61-4949-997d-1d370869181e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:00.219106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083050-8a61-4949-997d-1d370869181e admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:00.219788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8083050-8a61-4949-997d-1d370869181e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:00.221743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8083050-8a61-4949-997d-1d370869181e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:00.332809 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1139/2256] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:58:00 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:00.333496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00442157-076f-4d01-be4b-66d49351e4b6 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:00.352546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00442157-076f-4d01-be4b-66d49351e4b6 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:00.357850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb979ed-7993-4206-b5b4-c3bdd16c988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:00.361443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb979ed-7993-4206-b5b4-c3bdd16c988e admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:00.365390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb979ed-7993-4206-b5b4-c3bdd16c988e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:00.365716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb979ed-7993-4206-b5b4-c3bdd16c988e admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:00.366481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb979ed-7993-4206-b5b4-c3bdd16c988e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:00.368214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb979ed-7993-4206-b5b4-c3bdd16c988e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:00.380392 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1118/2257] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:00.386066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8083050-8a61-4949-997d-1d370869181e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:00.397218 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8083050-8a61-4949-997d-1d370869181e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:00.402030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173a28b-11ef-4edf-b36a-4fb1f9d54aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:00.403080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173a28b-11ef-4edf-b36a-4fb1f9d54aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:00.403336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173a28b-11ef-4edf-b36a-4fb1f9d54aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:00.403620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173a28b-11ef-4edf-b36a-4fb1f9d54aa7 admin admin] PATH_INFO: `/v3/projects/9dd164b80b2d41e4af0a9c90c41928d1/users/c63c1a0699f04a7c86619f3b0c1245ab/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:00.420165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173a28b-11ef-4edf-b36a-4fb1f9d54aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd164b80b2d41e4af0a9c90c41928d1, user_id=c63c1a0699f04a7c86619f3b0c1245ab, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:00.422724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173a28b-11ef-4edf-b36a-4fb1f9d54aa7 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:00.487690 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1140/2258] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:00 2022] PUT /identity/v3/projects/9dd164b80b2d41e4af0a9c90c41928d1/users/c63c1a0699f04a7c86619f3b0c1245ab/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:00.493626 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb979ed-7993-4206-b5b4-c3bdd16c988e admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:00.501563 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb979ed-7993-4206-b5b4-c3bdd16c988e admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:00.578133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2187a0c-8e96-44cd-aa5c-336d3edbb1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:00.578964 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2187a0c-8e96-44cd-aa5c-336d3edbb1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:00.579354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2187a0c-8e96-44cd-aa5c-336d3edbb1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:00.579735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2187a0c-8e96-44cd-aa5c-336d3edbb1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:00.580795 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2187a0c-8e96-44cd-aa5c-336d3edbb1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:00.582897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2187a0c-8e96-44cd-aa5c-336d3edbb1d2 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:00.600225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1119/2259] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:00.600570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173a28b-11ef-4edf-b36a-4fb1f9d54aa7 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:00.612746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173a28b-11ef-4edf-b36a-4fb1f9d54aa7 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:00.617702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c206592-5345-4f2e-9d60-e75effb185ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:00.620323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c206592-5345-4f2e-9d60-e75effb185ef admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:00.620323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c206592-5345-4f2e-9d60-e75effb185ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:00.620323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c206592-5345-4f2e-9d60-e75effb185ef admin admin] PATH_INFO: `/v3/projects/9dd164b80b2d41e4af0a9c90c41928d1/users/c63c1a0699f04a7c86619f3b0c1245ab/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:00.622468 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c206592-5345-4f2e-9d60-e75effb185ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd164b80b2d41e4af0a9c90c41928d1, user_id=c63c1a0699f04a7c86619f3b0c1245ab, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:00.626192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c206592-5345-4f2e-9d60-e75effb185ef admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:00.645998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1141/2260] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:00 2022] PUT /identity/v3/projects/9dd164b80b2d41e4af0a9c90c41928d1/users/c63c1a0699f04a7c86619f3b0c1245ab/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:58:08.152242 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2187a0c-8e96-44cd-aa5c-336d3edbb1d2 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:08.155738 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d2187a0c-8e96-44cd-aa5c-336d3edbb1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:08.174343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2187a0c-8e96-44cd-aa5c-336d3edbb1d2 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:08.261473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ea55c-71a4-456c-88f3-f19307e0d94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:08.263002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ea55c-71a4-456c-88f3-f19307e0d94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:08.263292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ea55c-71a4-456c-88f3-f19307e0d94f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:08.263571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ea55c-71a4-456c-88f3-f19307e0d94f admin admin] PATH_INFO: `/v3/users/40084b453941407489dc6ae3f3f01243` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:08.265190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ea55c-71a4-456c-88f3-f19307e0d94f admin admin] RBAC: Authorizing `identity:delete_user(user_id=40084b453941407489dc6ae3f3f01243)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:08.267231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ea55c-71a4-456c-88f3-f19307e0d94f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:08.496944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5a6ea55c-71a4-456c-88f3-f19307e0d94f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40084b453941407489dc6ae3f3f01243'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:08.505545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:08.507350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5a6ea55c-71a4-456c-88f3-f19307e0d94f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40084b453941407489dc6ae3f3f01243'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:08.508254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5a6ea55c-71a4-456c-88f3-f19307e0d94f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40084b453941407489dc6ae3f3f01243'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:08.520924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5a6ea55c-71a4-456c-88f3-f19307e0d94f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40084b453941407489dc6ae3f3f01243'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:08.522108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5a6ea55c-71a4-456c-88f3-f19307e0d94f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40084b453941407489dc6ae3f3f01243'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:08.567641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b96587-45fe-4b66-b976-d4b96ecc90ad None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:08.573931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b96587-45fe-4b66-b976-d4b96ecc90ad None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:08.573931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b96587-45fe-4b66-b976-d4b96ecc90ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:08.595110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f1b96587-45fe-4b66-b976-d4b96ecc90ad None None] MFA Rules not processed for user `c63c1a0699f04a7c86619f3b0c1245ab`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:08.596299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f1b96587-45fe-4b66-b976-d4b96ecc90ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:08.610009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1120/2261] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:58:08 2022] DELETE /identity/v3/users/40084b453941407489dc6ae3f3f01243 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:08.610009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472f25e-102d-478f-9a87-06c0aafac400 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:08.610009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472f25e-102d-478f-9a87-06c0aafac400 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:08.610009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472f25e-102d-478f-9a87-06c0aafac400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:08.628631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-5472f25e-102d-478f-9a87-06c0aafac400 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:08.629898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5472f25e-102d-478f-9a87-06c0aafac400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:08.678288 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1142/2262] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:58:08.699488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1121/2263] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:58:08.702571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b96587-45fe-4b66-b976-d4b96ecc90ad None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:08.704035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f1b96587-45fe-4b66-b976-d4b96ecc90ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:08.709004 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472f25e-102d-478f-9a87-06c0aafac400 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:08.711130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5472f25e-102d-478f-9a87-06c0aafac400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:08.772722 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472f25e-102d-478f-9a87-06c0aafac400 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:08.778049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26a3f8-8c7a-49c3-8940-cccfa40303ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:08.779076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26a3f8-8c7a-49c3-8940-cccfa40303ef service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:08.779604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26a3f8-8c7a-49c3-8940-cccfa40303ef service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:08.779965 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26a3f8-8c7a-49c3-8940-cccfa40303ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:08.780424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:08.790902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26a3f8-8c7a-49c3-8940-cccfa40303ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:08.792433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26a3f8-8c7a-49c3-8940-cccfa40303ef service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:08.800632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b96587-45fe-4b66-b976-d4b96ecc90ad None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:08.802490 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1122/2264] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:08.871869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9ea44-13bc-4ec1-9f32-2ff02047af48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:08.872532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9ea44-13bc-4ec1-9f32-2ff02047af48 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:08.872819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9ea44-13bc-4ec1-9f32-2ff02047af48 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:08.873097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9ea44-13bc-4ec1-9f32-2ff02047af48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:08.881267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ddd9ea44-13bc-4ec1-9f32-2ff02047af48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:08.949791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9ea44-13bc-4ec1-9f32-2ff02047af48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:08.951552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9ea44-13bc-4ec1-9f32-2ff02047af48 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:08.961435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1143/2265] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:58:09.587570 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26a3f8-8c7a-49c3-8940-cccfa40303ef service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:09.590645 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-df26a3f8-8c7a-49c3-8940-cccfa40303ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:09.609775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26a3f8-8c7a-49c3-8940-cccfa40303ef service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:09.617614 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:09.618703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:09.619177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:09.619688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] PATH_INFO: `/v3/projects/f77b565d29b44bedab9c0ac1b8cc4c63` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:09.621233 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f77b565d29b44bedab9c0ac1b8cc4c63)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:09.629637 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:09.815316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f77b565d29b44bedab9c0ac1b8cc4c63', 'request_id': 'req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:09.894378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f77b565d29b44bedab9c0ac1b8cc4c63', 'request_id': 'req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:09.901662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f77b565d29b44bedab9c0ac1b8cc4c63', 'request_id': 'req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:09.918471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:09.935947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] The token cache is being invalidate because project f77b565d29b44bedab9c0ac1b8cc4c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:58:09.935947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:09.935947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-4b85785e-aba4-4dbf-a4d2-202c30c92dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:09.935947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1123/2266] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:58:09 2022] DELETE /identity/v3/projects/f77b565d29b44bedab9c0ac1b8cc4c63 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:09.941374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603caef4-bc89-4fd3-af61-831b40e0e4db None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:09.941845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603caef4-bc89-4fd3-af61-831b40e0e4db None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:09.942141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603caef4-bc89-4fd3-af61-831b40e0e4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:09.979175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-603caef4-bc89-4fd3-af61-831b40e0e4db None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:09.980866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-603caef4-bc89-4fd3-af61-831b40e0e4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:10.065996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1144/2267] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:10.074495 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f3c0d-ec77-4c47-8819-f792c4ef6719 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:10.074786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f3c0d-ec77-4c47-8819-f792c4ef6719 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:10.075065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f3c0d-ec77-4c47-8819-f792c4ef6719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:10.094555 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-a26f3c0d-ec77-4c47-8819-f792c4ef6719 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:10.095993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a26f3c0d-ec77-4c47-8819-f792c4ef6719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:10.104055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1124/2268] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:10.108756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603caef4-bc89-4fd3-af61-831b40e0e4db None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:10.129171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603caef4-bc89-4fd3-af61-831b40e0e4db None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:10.133979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0c8f1-1c10-497a-9a61-964e31f657de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:10.134644 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0c8f1-1c10-497a-9a61-964e31f657de admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:10.134925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0c8f1-1c10-497a-9a61-964e31f657de admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:10.135788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0c8f1-1c10-497a-9a61-964e31f657de admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:10.136576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0c8f1-1c10-497a-9a61-964e31f657de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:58:10.136887 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0c8f1-1c10-497a-9a61-964e31f657de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:10.138653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0c8f1-1c10-497a-9a61-964e31f657de admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:10.156611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1145/2269] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:58:10.158870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f3c0d-ec77-4c47-8819-f792c4ef6719 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:10.169079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f3c0d-ec77-4c47-8819-f792c4ef6719 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:10.174085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcfd97-ecf1-4989-9305-97e466056385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:10.174746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcfd97-ecf1-4989-9305-97e466056385 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:10.175033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcfd97-ecf1-4989-9305-97e466056385 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:10.175310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcfd97-ecf1-4989-9305-97e466056385 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:10.176548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcfd97-ecf1-4989-9305-97e466056385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:10.177985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcfd97-ecf1-4989-9305-97e466056385 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:10.296124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1125/2270] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:58:10 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:10.300982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0c8f1-1c10-497a-9a61-964e31f657de admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:10.321575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0c8f1-1c10-497a-9a61-964e31f657de admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:10.353080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:10.427664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3f1f7-9ab0-45aa-b594-c96d32f180d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:10.429328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3f1f7-9ab0-45aa-b594-c96d32f180d8 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:10.430267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3f1f7-9ab0-45aa-b594-c96d32f180d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:10.430559 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3f1f7-9ab0-45aa-b594-c96d32f180d8 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:10.431271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3f1f7-9ab0-45aa-b594-c96d32f180d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:10.436888 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3f1f7-9ab0-45aa-b594-c96d32f180d8 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:10.590332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1146/2271] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:58:10 2022] POST /identity/v3/users => generated 291 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:10.591769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcfd97-ecf1-4989-9305-97e466056385 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:10.609666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcfd97-ecf1-4989-9305-97e466056385 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:10.624440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b21c7c-e494-4b11-858f-d1daa7e83d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:10.625652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b21c7c-e494-4b11-858f-d1daa7e83d1c admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:10.629694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b21c7c-e494-4b11-858f-d1daa7e83d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:10.629694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b21c7c-e494-4b11-858f-d1daa7e83d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:10.629694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b21c7c-e494-4b11-858f-d1daa7e83d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:10.629694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b21c7c-e494-4b11-858f-d1daa7e83d1c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:10.641468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1126/2272] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:10.645730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3f1f7-9ab0-45aa-b594-c96d32f180d8 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:10.657217 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3f1f7-9ab0-45aa-b594-c96d32f180d8 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:10.665561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe9c41-d93a-43f0-b9a3-73c1fd81d28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:10.666581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9c41-d93a-43f0-b9a3-73c1fd81d28c admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:10.666935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9c41-d93a-43f0-b9a3-73c1fd81d28c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:10.667273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9c41-d93a-43f0-b9a3-73c1fd81d28c admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:10.668182 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe9c41-d93a-43f0-b9a3-73c1fd81d28c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:10.670492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe9c41-d93a-43f0-b9a3-73c1fd81d28c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:10.703145 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1147/2273] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:10.705718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b21c7c-e494-4b11-858f-d1daa7e83d1c admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:10.716869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b21c7c-e494-4b11-858f-d1daa7e83d1c admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:10.721632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db73e90-da98-43cb-855c-76d04c1debf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:10.722611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db73e90-da98-43cb-855c-76d04c1debf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:10.722994 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db73e90-da98-43cb-855c-76d04c1debf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:10.723365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db73e90-da98-43cb-855c-76d04c1debf7 admin admin] PATH_INFO: `/v3/projects/92b9668d4c28498587fe6f0de16c7cad/users/d270f037d46840e9bd8e685e549c23e4/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:10.740980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db73e90-da98-43cb-855c-76d04c1debf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b9668d4c28498587fe6f0de16c7cad, user_id=d270f037d46840e9bd8e685e549c23e4, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:10.744034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db73e90-da98-43cb-855c-76d04c1debf7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:10.834864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1127/2274] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:10 2022] PUT /identity/v3/projects/92b9668d4c28498587fe6f0de16c7cad/users/d270f037d46840e9bd8e685e549c23e4/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:58:10.865552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdb2cb-b473-4a94-a8d0-391a7a0a1cab None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:10.865552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdb2cb-b473-4a94-a8d0-391a7a0a1cab None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:10.865552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdb2cb-b473-4a94-a8d0-391a7a0a1cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:10.885836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-26bdb2cb-b473-4a94-a8d0-391a7a0a1cab None None] MFA Rules not processed for user `d270f037d46840e9bd8e685e549c23e4`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:10.887332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-26bdb2cb-b473-4a94-a8d0-391a7a0a1cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:10.981615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1148/2275] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:10.982773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db73e90-da98-43cb-855c-76d04c1debf7 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:11.086348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db73e90-da98-43cb-855c-76d04c1debf7 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:11.091146 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73474ce-4732-4d3c-90b7-b5973ad3baa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:11.091807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73474ce-4732-4d3c-90b7-b5973ad3baa9 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:11.092090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73474ce-4732-4d3c-90b7-b5973ad3baa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:11.092371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73474ce-4732-4d3c-90b7-b5973ad3baa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:11.093065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73474ce-4732-4d3c-90b7-b5973ad3baa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:11.095196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73474ce-4732-4d3c-90b7-b5973ad3baa9 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:11.223078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1128/2276] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:58:10 2022] POST /identity/v3/projects => generated 383 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:11.227038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdb2cb-b473-4a94-a8d0-391a7a0a1cab None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:11.240756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdb2cb-b473-4a94-a8d0-391a7a0a1cab None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:11.316551 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dbc91-e132-4249-99c6-e9adebe2c622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:11.317464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dbc91-e132-4249-99c6-e9adebe2c622 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:11.317756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dbc91-e132-4249-99c6-e9adebe2c622 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:11.318040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dbc91-e132-4249-99c6-e9adebe2c622 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:11.318957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08dbc91-e132-4249-99c6-e9adebe2c622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:11.321266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08dbc91-e132-4249-99c6-e9adebe2c622 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:11.418306 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1149/2277] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:58:11 2022] POST /identity/v3/users => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:11.419776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73474ce-4732-4d3c-90b7-b5973ad3baa9 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:11.441750 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73474ce-4732-4d3c-90b7-b5973ad3baa9 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:11.455760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946a580-a2f9-497c-99c6-fd89b9e977fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:11.455760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946a580-a2f9-497c-99c6-fd89b9e977fe admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:11.455760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946a580-a2f9-497c-99c6-fd89b9e977fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:11.455760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946a580-a2f9-497c-99c6-fd89b9e977fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:11.456211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946a580-a2f9-497c-99c6-fd89b9e977fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:11.456211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946a580-a2f9-497c-99c6-fd89b9e977fe admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:11.472063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dbc91-e132-4249-99c6-e9adebe2c622 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:11.480997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1129/2278] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:11.482422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dbc91-e132-4249-99c6-e9adebe2c622 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:11.488556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0ae73-f8a0-4fda-9fc8-fc1dbae422f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:11.489500 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0ae73-f8a0-4fda-9fc8-fc1dbae422f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:11.489786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0ae73-f8a0-4fda-9fc8-fc1dbae422f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:11.490064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0ae73-f8a0-4fda-9fc8-fc1dbae422f8 admin admin] PATH_INFO: `/v3/projects/acb171458204470b908b7f2807bd05aa/users/2b0b4a9adaa940d99a7e0255460fe723/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:11.505940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0ae73-f8a0-4fda-9fc8-fc1dbae422f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb171458204470b908b7f2807bd05aa, user_id=2b0b4a9adaa940d99a7e0255460fe723, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:11.508584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0ae73-f8a0-4fda-9fc8-fc1dbae422f8 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:11.623784 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:11.669173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1150/2279] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:11 2022] PUT /identity/v3/projects/acb171458204470b908b7f2807bd05aa/users/2b0b4a9adaa940d99a7e0255460fe723/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:58:11.670159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946a580-a2f9-497c-99c6-fd89b9e977fe admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:11.700966 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946a580-a2f9-497c-99c6-fd89b9e977fe admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:11.870684 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54562bed-90a7-4a9c-961c-21ed4911e052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:11.871264 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54562bed-90a7-4a9c-961c-21ed4911e052 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:11.871547 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54562bed-90a7-4a9c-961c-21ed4911e052 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:11.871920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54562bed-90a7-4a9c-961c-21ed4911e052 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:11.872687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54562bed-90a7-4a9c-961c-21ed4911e052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:11.874328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54562bed-90a7-4a9c-961c-21ed4911e052 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:11.925432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0ae73-f8a0-4fda-9fc8-fc1dbae422f8 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:11.926767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1130/2280] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:11.938083 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0ae73-f8a0-4fda-9fc8-fc1dbae422f8 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:11.945032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cec2d2-739c-4898-8094-a63b8348961e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:11.950404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cec2d2-739c-4898-8094-a63b8348961e admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:11.950404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cec2d2-739c-4898-8094-a63b8348961e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:11.950404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cec2d2-739c-4898-8094-a63b8348961e admin admin] PATH_INFO: `/v3/projects/acb171458204470b908b7f2807bd05aa/users/2b0b4a9adaa940d99a7e0255460fe723/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:11.950404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cec2d2-739c-4898-8094-a63b8348961e admin admin] RBAC: Authorizing `identity:create_grant(project_id=acb171458204470b908b7f2807bd05aa, user_id=2b0b4a9adaa940d99a7e0255460fe723, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:11.951544 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cec2d2-739c-4898-8094-a63b8348961e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:11.967322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1151/2281] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:11 2022] PUT /identity/v3/projects/acb171458204470b908b7f2807bd05aa/users/2b0b4a9adaa940d99a7e0255460fe723/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:58:11.977690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb3e72-595f-4960-89d2-7c3dfaeeef30 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:11.978192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb3e72-595f-4960-89d2-7c3dfaeeef30 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:11.979644 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb3e72-595f-4960-89d2-7c3dfaeeef30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:11.988752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:12.017107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-53cb3e72-595f-4960-89d2-7c3dfaeeef30 None None] MFA Rules not processed for user `2b0b4a9adaa940d99a7e0255460fe723`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:12.018747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-53cb3e72-595f-4960-89d2-7c3dfaeeef30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:12.157327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1131/2282] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:12.182435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cec2d2-739c-4898-8094-a63b8348961e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:12.184900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-75cec2d2-739c-4898-8094-a63b8348961e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:12.288217 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cec2d2-739c-4898-8094-a63b8348961e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:12.302867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f261cc6-e419-46b1-9768-5cfdca8f050b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:12.309335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f261cc6-e419-46b1-9768-5cfdca8f050b service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:12.309335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f261cc6-e419-46b1-9768-5cfdca8f050b service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:12.313052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f261cc6-e419-46b1-9768-5cfdca8f050b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:12.328178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f261cc6-e419-46b1-9768-5cfdca8f050b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:12.329900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f261cc6-e419-46b1-9768-5cfdca8f050b service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:12.373461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1152/2283] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:58:12.543033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb3e72-595f-4960-89d2-7c3dfaeeef30 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:12.555787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb3e72-595f-4960-89d2-7c3dfaeeef30 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:12.561340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292bc2e9-2fba-42df-8453-e0ade58a33d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:12.562015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bc2e9-2fba-42df-8453-e0ade58a33d1 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:12.562305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bc2e9-2fba-42df-8453-e0ade58a33d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:12.562588 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292bc2e9-2fba-42df-8453-e0ade58a33d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:12.637346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bc2e9-2fba-42df-8453-e0ade58a33d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:12.639042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292bc2e9-2fba-42df-8453-e0ade58a33d1 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:12.660000 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1132/2284] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:13.394770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f261cc6-e419-46b1-9768-5cfdca8f050b service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:13.496346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f261cc6-e419-46b1-9768-5cfdca8f050b service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:13.501605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:13.502582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:13.502874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:13.503151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] PATH_INFO: `/v3/users/d270f037d46840e9bd8e685e549c23e4` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:13.504425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=d270f037d46840e9bd8e685e549c23e4)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:13.506422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:13.706489 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd270f037d46840e9bd8e685e549c23e4'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:13.707015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd270f037d46840e9bd8e685e549c23e4'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:13.715084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd270f037d46840e9bd8e685e549c23e4'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:13.715896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd270f037d46840e9bd8e685e549c23e4'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:13.747435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd270f037d46840e9bd8e685e549c23e4'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:13.897932 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1153/2285] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:58:13 2022] DELETE /identity/v3/users/d270f037d46840e9bd8e685e549c23e4 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:13.904562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599afd-360c-48f2-bc7d-9fa1203dae7b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:13.905104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599afd-360c-48f2-bc7d-9fa1203dae7b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:13.905104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599afd-360c-48f2-bc7d-9fa1203dae7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:14.005003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d8599afd-360c-48f2-bc7d-9fa1203dae7b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:14.007164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d8599afd-360c-48f2-bc7d-9fa1203dae7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:14.077818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:14.146732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1133/2286] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:58:14.157433 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:14.160521 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:14.168900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:14.324895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1c0bd-0c9a-4daf-ae8c-9c9bad2406ce admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:14.330030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ac907-8a2d-4f9e-add0-ebcbdc1d3d7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:14.330818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ac907-8a2d-4f9e-add0-ebcbdc1d3d7e service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:14.331105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ac907-8a2d-4f9e-add0-ebcbdc1d3d7e service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:14.331378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ac907-8a2d-4f9e-add0-ebcbdc1d3d7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:14.341043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ac907-8a2d-4f9e-add0-ebcbdc1d3d7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:14.343069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ac907-8a2d-4f9e-add0-ebcbdc1d3d7e service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:14.352472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1154/2287] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:15.081187 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8599afd-360c-48f2-bc7d-9fa1203dae7b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:15.082769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d8599afd-360c-48f2-bc7d-9fa1203dae7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:15.126351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8599afd-360c-48f2-bc7d-9fa1203dae7b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:15.139491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:15.140377 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:15.140657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:15.140950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] PATH_INFO: `/v3/projects/92b9668d4c28498587fe6f0de16c7cad` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:15.142398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=92b9668d4c28498587fe6f0de16c7cad)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:15.144303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:15.261909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92b9668d4c28498587fe6f0de16c7cad', 'request_id': 'req-08592439-d249-424a-9bd6-c027043a35ae'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:15.380866 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92b9668d4c28498587fe6f0de16c7cad', 'request_id': 'req-08592439-d249-424a-9bd6-c027043a35ae'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:15.388808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92b9668d4c28498587fe6f0de16c7cad', 'request_id': 'req-08592439-d249-424a-9bd6-c027043a35ae'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:15.427322 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] The token cache is being invalidate because project 92b9668d4c28498587fe6f0de16c7cad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:58:15.427855 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:15.428650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-08592439-d249-424a-9bd6-c027043a35ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:15.431025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1134/2288] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:58:15 2022] DELETE /identity/v3/projects/92b9668d4c28498587fe6f0de16c7cad => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:15.434723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ac907-8a2d-4f9e-add0-ebcbdc1d3d7e service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:15.447316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ac907-8a2d-4f9e-add0-ebcbdc1d3d7e service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:15.534192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:15.535171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:15.535463 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:15.539112 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] PATH_INFO: `/v3/users/2b0b4a9adaa940d99a7e0255460fe723` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:15.539112 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b0b4a9adaa940d99a7e0255460fe723)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:15.539112 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:15.782940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b0b4a9adaa940d99a7e0255460fe723'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:15.783666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b0b4a9adaa940d99a7e0255460fe723'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:15.791673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b0b4a9adaa940d99a7e0255460fe723'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:15.792564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b0b4a9adaa940d99a7e0255460fe723'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:15.806634 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:15.809278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b0b4a9adaa940d99a7e0255460fe723'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:15.900879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1155/2289] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:58:15 2022] DELETE /identity/v3/users/2b0b4a9adaa940d99a7e0255460fe723 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:58:16.371881 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f59e0-c66f-4356-a96e-1f08fc8303f4 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:16.372901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f59e0-c66f-4356-a96e-1f08fc8303f4 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:16.373252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f59e0-c66f-4356-a96e-1f08fc8303f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:16.402352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-607f59e0-c66f-4356-a96e-1f08fc8303f4 None None] MFA Rules not processed for user `7310c631282c495a83e17dbdbc6d2ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:16.404484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-607f59e0-c66f-4356-a96e-1f08fc8303f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:16.478420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1135/2290] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:16.596999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:16.598690 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:16.691542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102f3a8-1e4f-4cc9-8229-b122adcde4ad admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:16.696454 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:16.697396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:16.697716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:16.698000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] PATH_INFO: `/v3/projects/acb171458204470b908b7f2807bd05aa` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:16.700173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=acb171458204470b908b7f2807bd05aa)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:16.702442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:16.890021 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acb171458204470b908b7f2807bd05aa', 'request_id': 'req-8228cfea-6b5e-4cc8-8587-30d592671f9d'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:16.992444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acb171458204470b908b7f2807bd05aa', 'request_id': 'req-8228cfea-6b5e-4cc8-8587-30d592671f9d'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:17.001952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acb171458204470b908b7f2807bd05aa', 'request_id': 'req-8228cfea-6b5e-4cc8-8587-30d592671f9d'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:17.036196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] The token cache is being invalidate because project acb171458204470b908b7f2807bd05aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:58:17.036512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:17.038447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8228cfea-6b5e-4cc8-8587-30d592671f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:17.053937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1156/2291] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:58:16 2022] DELETE /identity/v3/projects/acb171458204470b908b7f2807bd05aa => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:17.060906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a003f40-3b13-4ab8-a151-6d232497e1a5 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:17.060906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a003f40-3b13-4ab8-a151-6d232497e1a5 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:17.060906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a003f40-3b13-4ab8-a151-6d232497e1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:17.111628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2a003f40-3b13-4ab8-a151-6d232497e1a5 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:17.112794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2a003f40-3b13-4ab8-a151-6d232497e1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:17.163916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:17.222619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1136/2292] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:17.230597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead98cc-f397-47db-a27a-aa75ae13897f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:17.230887 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead98cc-f397-47db-a27a-aa75ae13897f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:17.231733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead98cc-f397-47db-a27a-aa75ae13897f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:17.261287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-aead98cc-f397-47db-a27a-aa75ae13897f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:17.263051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-aead98cc-f397-47db-a27a-aa75ae13897f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:17.273280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1157/2293] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:17.278413 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a003f40-3b13-4ab8-a151-6d232497e1a5 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:17.305095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a003f40-3b13-4ab8-a151-6d232497e1a5 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:17.335864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7df43e-4c95-4281-b6cf-42150dc562b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:17.335864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7df43e-4c95-4281-b6cf-42150dc562b9 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:17.335864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7df43e-4c95-4281-b6cf-42150dc562b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:17.335864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7df43e-4c95-4281-b6cf-42150dc562b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:17.336287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7df43e-4c95-4281-b6cf-42150dc562b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:58:17.336641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7df43e-4c95-4281-b6cf-42150dc562b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:17.338493 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7df43e-4c95-4281-b6cf-42150dc562b9 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:17.370867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1137/2294] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:58:17.376206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead98cc-f397-47db-a27a-aa75ae13897f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:17.397228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead98cc-f397-47db-a27a-aa75ae13897f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:17.406712 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff608587-12e2-4fe6-8d17-0acbf89ec0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:17.417053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff608587-12e2-4fe6-8d17-0acbf89ec0d8 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:17.417388 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff608587-12e2-4fe6-8d17-0acbf89ec0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:17.417707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff608587-12e2-4fe6-8d17-0acbf89ec0d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:17.418433 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff608587-12e2-4fe6-8d17-0acbf89ec0d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:17.421269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff608587-12e2-4fe6-8d17-0acbf89ec0d8 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:17.588052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1158/2295] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:58:17 2022] POST /identity/v3/projects => generated 375 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:17.591167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7df43e-4c95-4281-b6cf-42150dc562b9 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:17.607008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7df43e-4c95-4281-b6cf-42150dc562b9 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:17.736370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae11aa-ea34-4dd1-aef0-e132d7a7cc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:17.737907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae11aa-ea34-4dd1-aef0-e132d7a7cc0a admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:17.738359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae11aa-ea34-4dd1-aef0-e132d7a7cc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:17.738888 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae11aa-ea34-4dd1-aef0-e132d7a7cc0a admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:17.740274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae11aa-ea34-4dd1-aef0-e132d7a7cc0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:17.745656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae11aa-ea34-4dd1-aef0-e132d7a7cc0a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:17.859187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff608587-12e2-4fe6-8d17-0acbf89ec0d8 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:17.859789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1138/2296] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:58:17 2022] POST /identity/v3/users => generated 288 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:17.878293 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff608587-12e2-4fe6-8d17-0acbf89ec0d8 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:17.889685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe998483-2382-42eb-8d50-f9b9c052912b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:17.890374 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe998483-2382-42eb-8d50-f9b9c052912b admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:17.890659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe998483-2382-42eb-8d50-f9b9c052912b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:17.891104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe998483-2382-42eb-8d50-f9b9c052912b admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:17.891978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe998483-2382-42eb-8d50-f9b9c052912b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:17.894305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe998483-2382-42eb-8d50-f9b9c052912b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:17.925015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1159/2297] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:17.933757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae11aa-ea34-4dd1-aef0-e132d7a7cc0a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:17.941839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae11aa-ea34-4dd1-aef0-e132d7a7cc0a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:17.956696 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684c45d-23a3-4e85-a8f8-087bae154006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:17.957489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684c45d-23a3-4e85-a8f8-087bae154006 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:17.963420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684c45d-23a3-4e85-a8f8-087bae154006 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:17.963420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684c45d-23a3-4e85-a8f8-087bae154006 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:17.963420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684c45d-23a3-4e85-a8f8-087bae154006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:17.963420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9684c45d-23a3-4e85-a8f8-087bae154006 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:17.976177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1139/2298] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:17.978062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe998483-2382-42eb-8d50-f9b9c052912b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:17.996572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe998483-2382-42eb-8d50-f9b9c052912b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:18.003224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ff115-5eea-4f0f-885a-e35259c33c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:18.004230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ff115-5eea-4f0f-885a-e35259c33c5f admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:18.006899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ff115-5eea-4f0f-885a-e35259c33c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:18.007937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ff115-5eea-4f0f-885a-e35259c33c5f admin admin] PATH_INFO: `/v3/projects/8d48a2ba51f24ce1acca365be720aa68/users/f870d738fb9346be841477fe9af19815/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:18.027592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ff115-5eea-4f0f-885a-e35259c33c5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d48a2ba51f24ce1acca365be720aa68, user_id=f870d738fb9346be841477fe9af19815, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:18.030565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ff115-5eea-4f0f-885a-e35259c33c5f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:18.172567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1160/2299] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:17 2022] PUT /identity/v3/projects/8d48a2ba51f24ce1acca365be720aa68/users/f870d738fb9346be841477fe9af19815/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:18.188689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd6be2-c7ab-471a-881f-aa4d753e85ad None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:18.189012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd6be2-c7ab-471a-881f-aa4d753e85ad None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:18.189331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abd6be2-c7ab-471a-881f-aa4d753e85ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:18.282921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1abd6be2-c7ab-471a-881f-aa4d753e85ad None None] MFA Rules not processed for user `f870d738fb9346be841477fe9af19815`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:18.284189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1abd6be2-c7ab-471a-881f-aa4d753e85ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:18.414062 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1140/2300] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:18.415232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ff115-5eea-4f0f-885a-e35259c33c5f admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:18.458746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:18.534850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ff115-5eea-4f0f-885a-e35259c33c5f admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:18.543166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63891be7-b868-4753-bea9-3c12f7326a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:18.544038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63891be7-b868-4753-bea9-3c12f7326a9d admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:18.544277 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63891be7-b868-4753-bea9-3c12f7326a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:18.544755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63891be7-b868-4753-bea9-3c12f7326a9d admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:18.545593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63891be7-b868-4753-bea9-3c12f7326a9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:18.548403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63891be7-b868-4753-bea9-3c12f7326a9d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:18.649916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1161/2301] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:58:18 2022] POST /identity/v3/projects => generated 373 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:18.654660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd6be2-c7ab-471a-881f-aa4d753e85ad None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:18.670663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abd6be2-c7ab-471a-881f-aa4d753e85ad None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:18.706511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:18.739263 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bccad-7073-43e8-b86a-aa29225de807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:18.739953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bccad-7073-43e8-b86a-aa29225de807 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:18.740238 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bccad-7073-43e8-b86a-aa29225de807 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:18.740780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bccad-7073-43e8-b86a-aa29225de807 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:18.741459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bccad-7073-43e8-b86a-aa29225de807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:18.743674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bccad-7073-43e8-b86a-aa29225de807 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:18.851957 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1141/2302] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:58:18 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:18.853314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63891be7-b868-4753-bea9-3c12f7326a9d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:18.864200 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63891be7-b868-4753-bea9-3c12f7326a9d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:18.869851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77206f5-4902-4b0d-8dda-29a9af39b93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:18.870929 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77206f5-4902-4b0d-8dda-29a9af39b93d admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:18.871218 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77206f5-4902-4b0d-8dda-29a9af39b93d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:18.871493 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77206f5-4902-4b0d-8dda-29a9af39b93d admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:18.872303 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77206f5-4902-4b0d-8dda-29a9af39b93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:18.873944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77206f5-4902-4b0d-8dda-29a9af39b93d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:18.896959 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1162/2303] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:18.906173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bccad-7073-43e8-b86a-aa29225de807 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:18.915759 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bccad-7073-43e8-b86a-aa29225de807 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:18.925528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21921e19-b35d-4cef-afe2-204ddaf48d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:18.926563 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21921e19-b35d-4cef-afe2-204ddaf48d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:18.926944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21921e19-b35d-4cef-afe2-204ddaf48d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:18.927292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21921e19-b35d-4cef-afe2-204ddaf48d78 admin admin] PATH_INFO: `/v3/projects/6503ce596c8841f98876a2a1c5a789e3/users/ad833e56a1d84d50810a43ecbba2b858/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:18.945424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21921e19-b35d-4cef-afe2-204ddaf48d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6503ce596c8841f98876a2a1c5a789e3, user_id=ad833e56a1d84d50810a43ecbba2b858, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:18.949728 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21921e19-b35d-4cef-afe2-204ddaf48d78 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:19.030979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1142/2304] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:18 2022] PUT /identity/v3/projects/6503ce596c8841f98876a2a1c5a789e3/users/ad833e56a1d84d50810a43ecbba2b858/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:19.032040 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77206f5-4902-4b0d-8dda-29a9af39b93d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:19.041899 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21921e19-b35d-4cef-afe2-204ddaf48d78 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:19.045232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77206f5-4902-4b0d-8dda-29a9af39b93d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:19.047227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-21921e19-b35d-4cef-afe2-204ddaf48d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:19.131271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d3ec0-a399-46a9-a32c-2686ef7df152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:19.131965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3ec0-a399-46a9-a32c-2686ef7df152 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:19.132256 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3ec0-a399-46a9-a32c-2686ef7df152 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:19.132576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d3ec0-a399-46a9-a32c-2686ef7df152 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:19.133368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d3ec0-a399-46a9-a32c-2686ef7df152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:19.134974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d3ec0-a399-46a9-a32c-2686ef7df152 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:19.140946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21921e19-b35d-4cef-afe2-204ddaf48d78 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:19.152136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d2bf4-5e7f-4859-8a68-f98c8cf17c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:19.152136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d2bf4-5e7f-4859-8a68-f98c8cf17c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:19.152336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1163/2305] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:19 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:19.152449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d2bf4-5e7f-4859-8a68-f98c8cf17c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:19.152449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d2bf4-5e7f-4859-8a68-f98c8cf17c16 admin admin] PATH_INFO: `/v3/users/5fb42e8a087d4d7983f8733c7eb337c7` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:19.152449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d2bf4-5e7f-4859-8a68-f98c8cf17c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb42e8a087d4d7983f8733c7eb337c7)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:19.152449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d2bf4-5e7f-4859-8a68-f98c8cf17c16 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:19.152759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d3ec0-a399-46a9-a32c-2686ef7df152 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:19.171280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d3ec0-a399-46a9-a32c-2686ef7df152 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:19.176818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259fca7-a5f3-4d07-90f8-998d9ffd2fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:19.177822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259fca7-a5f3-4d07-90f8-998d9ffd2fef admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:19.178110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259fca7-a5f3-4d07-90f8-998d9ffd2fef admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:19.178389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259fca7-a5f3-4d07-90f8-998d9ffd2fef admin admin] PATH_INFO: `/v3/projects/6503ce596c8841f98876a2a1c5a789e3/users/ad833e56a1d84d50810a43ecbba2b858/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:19.180657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7259fca7-a5f3-4d07-90f8-998d9ffd2fef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6503ce596c8841f98876a2a1c5a789e3, user_id=ad833e56a1d84d50810a43ecbba2b858, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:19.183961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7259fca7-a5f3-4d07-90f8-998d9ffd2fef admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:19.196530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1164/2306] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:19 2022] PUT /identity/v3/projects/6503ce596c8841f98876a2a1c5a789e3/users/ad833e56a1d84d50810a43ecbba2b858/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:58:19.207045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bd004-dd5c-4b9d-a3d5-4c3adccc0c40 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:19.207346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bd004-dd5c-4b9d-a3d5-4c3adccc0c40 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:19.207677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457bd004-dd5c-4b9d-a3d5-4c3adccc0c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:19.235777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-457bd004-dd5c-4b9d-a3d5-4c3adccc0c40 None None] MFA Rules not processed for user `ad833e56a1d84d50810a43ecbba2b858`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:19.237211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-457bd004-dd5c-4b9d-a3d5-4c3adccc0c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:19.311912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1165/2307] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:58:19.326680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bd004-dd5c-4b9d-a3d5-4c3adccc0c40 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:19.327312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-457bd004-dd5c-4b9d-a3d5-4c3adccc0c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:19.334632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-311d2bf4-5e7f-4859-8a68-f98c8cf17c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb42e8a087d4d7983f8733c7eb337c7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:19.350294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-311d2bf4-5e7f-4859-8a68-f98c8cf17c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb42e8a087d4d7983f8733c7eb337c7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:19.351186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-311d2bf4-5e7f-4859-8a68-f98c8cf17c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb42e8a087d4d7983f8733c7eb337c7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:19.374284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-311d2bf4-5e7f-4859-8a68-f98c8cf17c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb42e8a087d4d7983f8733c7eb337c7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:19.375231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-311d2bf4-5e7f-4859-8a68-f98c8cf17c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb42e8a087d4d7983f8733c7eb337c7'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:19.397855 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457bd004-dd5c-4b9d-a3d5-4c3adccc0c40 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:19.404535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df86ba8-1e52-4b7b-8c26-ae423891e684 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:19.406190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df86ba8-1e52-4b7b-8c26-ae423891e684 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:19.406593 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df86ba8-1e52-4b7b-8c26-ae423891e684 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:19.406945 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df86ba8-1e52-4b7b-8c26-ae423891e684 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:19.409350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9df86ba8-1e52-4b7b-8c26-ae423891e684 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:19.472782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:19.493567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df86ba8-1e52-4b7b-8c26-ae423891e684 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:19.495256 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df86ba8-1e52-4b7b-8c26-ae423891e684 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:19.508266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1143/2308] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:58:19 2022] DELETE /identity/v3/users/5fb42e8a087d4d7983f8733c7eb337c7 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:19.509722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1166/2309] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:58:19.513426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32e46e-4fc7-4db3-aeec-0491243b70ab None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:19.514577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32e46e-4fc7-4db3-aeec-0491243b70ab None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:19.515110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d32e46e-4fc7-4db3-aeec-0491243b70ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:19.551801 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6d32e46e-4fc7-4db3-aeec-0491243b70ab None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:19.553178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6d32e46e-4fc7-4db3-aeec-0491243b70ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:19.562785 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:19.630009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1144/2310] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:58:19.643113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df86ba8-1e52-4b7b-8c26-ae423891e684 service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:19.645170 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9df86ba8-1e52-4b7b-8c26-ae423891e684 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:19.730195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df86ba8-1e52-4b7b-8c26-ae423891e684 service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:19.806372 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e890c9-f572-4168-b2a8-02768f89cc1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:19.807663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e890c9-f572-4168-b2a8-02768f89cc1a service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:19.807919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e890c9-f572-4168-b2a8-02768f89cc1a service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:19.808348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e890c9-f572-4168-b2a8-02768f89cc1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:19.819666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e890c9-f572-4168-b2a8-02768f89cc1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:19.821147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e890c9-f572-4168-b2a8-02768f89cc1a service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:19.832026 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1167/2311] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:20.263404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32e46e-4fc7-4db3-aeec-0491243b70ab None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:20.265330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6d32e46e-4fc7-4db3-aeec-0491243b70ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:20.280808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d32e46e-4fc7-4db3-aeec-0491243b70ab None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:20.295034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:20.296460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:20.296853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:20.297918 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] PATH_INFO: `/v3/projects/2d8ec4168af342408ecb574617819cdd` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:20.299553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d8ec4168af342408ecb574617819cdd)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:20.311592 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:20.414523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d8ec4168af342408ecb574617819cdd', 'request_id': 'req-b523e98f-4e10-419b-bc1f-5ce207a78bc0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:20.506034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d8ec4168af342408ecb574617819cdd', 'request_id': 'req-b523e98f-4e10-419b-bc1f-5ce207a78bc0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:20.513603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d8ec4168af342408ecb574617819cdd', 'request_id': 'req-b523e98f-4e10-419b-bc1f-5ce207a78bc0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:20.526938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] The token cache is being invalidate because project 2d8ec4168af342408ecb574617819cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:58:20.526938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:20.527473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b523e98f-4e10-419b-bc1f-5ce207a78bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:20.529017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1145/2312] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:58:20 2022] DELETE /identity/v3/projects/2d8ec4168af342408ecb574617819cdd => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:20.534176 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a5ab-d3ba-4c31-a192-200b6ae7d34e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:20.534634 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a5ab-d3ba-4c31-a192-200b6ae7d34e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:20.534881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a5ab-d3ba-4c31-a192-200b6ae7d34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:20.554866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d0f3a5ab-d3ba-4c31-a192-200b6ae7d34e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:20.556151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d0f3a5ab-d3ba-4c31-a192-200b6ae7d34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:20.562342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:20.624431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1168/2313] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:20.633048 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dbc2f-7446-4525-8d6e-444d74f05334 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:20.633579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dbc2f-7446-4525-8d6e-444d74f05334 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:20.633861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dbc2f-7446-4525-8d6e-444d74f05334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:20.654461 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d31dbc2f-7446-4525-8d6e-444d74f05334 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:20.655928 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d31dbc2f-7446-4525-8d6e-444d74f05334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:20.665778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1146/2314] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:20.670584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3a5ab-d3ba-4c31-a192-200b6ae7d34e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:20.680854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3a5ab-d3ba-4c31-a192-200b6ae7d34e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:20.686797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f30e9-1352-48a5-8515-529a29477454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:20.687491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f30e9-1352-48a5-8515-529a29477454 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:20.687777 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f30e9-1352-48a5-8515-529a29477454 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:20.688053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f30e9-1352-48a5-8515-529a29477454 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:20.688774 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f30e9-1352-48a5-8515-529a29477454 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:58:20.692052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f30e9-1352-48a5-8515-529a29477454 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:20.692052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f30e9-1352-48a5-8515-529a29477454 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:20.715901 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1169/2315] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 18:58:20.717210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dbc2f-7446-4525-8d6e-444d74f05334 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:20.728366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dbc2f-7446-4525-8d6e-444d74f05334 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:20.735461 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7768cce-a9dc-45a5-a14a-60332f3777c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:20.736311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7768cce-a9dc-45a5-a14a-60332f3777c5 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:20.736788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7768cce-a9dc-45a5-a14a-60332f3777c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:20.737265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7768cce-a9dc-45a5-a14a-60332f3777c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:20.738101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7768cce-a9dc-45a5-a14a-60332f3777c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:20.740247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7768cce-a9dc-45a5-a14a-60332f3777c5 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:20.845808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1147/2316] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:58:20 2022] POST /identity/v3/projects => generated 371 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:20.850149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f30e9-1352-48a5-8515-529a29477454 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:20.861215 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f30e9-1352-48a5-8515-529a29477454 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:20.924587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f32bcb-1109-4a33-b24e-829b62a9a0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:20.925280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f32bcb-1109-4a33-b24e-829b62a9a0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:20.925563 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f32bcb-1109-4a33-b24e-829b62a9a0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:20.925838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f32bcb-1109-4a33-b24e-829b62a9a0f1 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:20.926607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f32bcb-1109-4a33-b24e-829b62a9a0f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:20.928528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f32bcb-1109-4a33-b24e-829b62a9a0f1 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:21.035150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1170/2317] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:58:20 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:21.039265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7768cce-a9dc-45a5-a14a-60332f3777c5 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:21.053749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7768cce-a9dc-45a5-a14a-60332f3777c5 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:21.062981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e76947-aa77-4c6e-ba15-f86e39b13f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:21.064193 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e76947-aa77-4c6e-ba15-f86e39b13f93 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:21.064504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e76947-aa77-4c6e-ba15-f86e39b13f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:21.065141 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e76947-aa77-4c6e-ba15-f86e39b13f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:21.065515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e76947-aa77-4c6e-ba15-f86e39b13f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:21.067135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e76947-aa77-4c6e-ba15-f86e39b13f93 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:21.085181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1148/2318] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:21.089460 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f32bcb-1109-4a33-b24e-829b62a9a0f1 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:21.100759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f32bcb-1109-4a33-b24e-829b62a9a0f1 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:21.109192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df90982-f8b2-4f0f-8c76-28d16e9adb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:21.110104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df90982-f8b2-4f0f-8c76-28d16e9adb71 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:21.110849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df90982-f8b2-4f0f-8c76-28d16e9adb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:21.111030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df90982-f8b2-4f0f-8c76-28d16e9adb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:21.112004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df90982-f8b2-4f0f-8c76-28d16e9adb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:21.113722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df90982-f8b2-4f0f-8c76-28d16e9adb71 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:21.130558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1171/2319] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:21.134782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e76947-aa77-4c6e-ba15-f86e39b13f93 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:21.147987 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e76947-aa77-4c6e-ba15-f86e39b13f93 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:21.152820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017dd638-f6eb-4110-a3e5-8a835b61591d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:21.153927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dd638-f6eb-4110-a3e5-8a835b61591d admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:21.154165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dd638-f6eb-4110-a3e5-8a835b61591d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:21.157522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dd638-f6eb-4110-a3e5-8a835b61591d admin admin] PATH_INFO: `/v3/projects/7969c68692884392ad5321b686df4801/users/91745b9228804ee1844aa498ad3d3ae9/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:21.202835 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017dd638-f6eb-4110-a3e5-8a835b61591d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7969c68692884392ad5321b686df4801, user_id=91745b9228804ee1844aa498ad3d3ae9, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:21.205770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017dd638-f6eb-4110-a3e5-8a835b61591d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:21.324039 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1149/2320] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:21 2022] PUT /identity/v3/projects/7969c68692884392ad5321b686df4801/users/91745b9228804ee1844aa498ad3d3ae9/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:21.334271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b352a-2449-4992-8256-bd96004e2c53 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:21.334566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b352a-2449-4992-8256-bd96004e2c53 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:21.334834 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b352a-2449-4992-8256-bd96004e2c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:21.364244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-5a5b352a-2449-4992-8256-bd96004e2c53 None None] MFA Rules not processed for user `91745b9228804ee1844aa498ad3d3ae9`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:21.365501 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5a5b352a-2449-4992-8256-bd96004e2c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:21.447247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1172/2321] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:21.459743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017dd638-f6eb-4110-a3e5-8a835b61591d admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:21.461675 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-017dd638-f6eb-4110-a3e5-8a835b61591d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:21.505226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:21.542388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017dd638-f6eb-4110-a3e5-8a835b61591d admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:21.547156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b0aea-ce08-4dd1-abbe-e69e69ec7178 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17b7a7128d64b2ba548add02d3ec264', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:21.547912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b0aea-ce08-4dd1-abbe-e69e69ec7178 service glance] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:21.548265 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b0aea-ce08-4dd1-abbe-e69e69ec7178 service glance] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:21.549214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b0aea-ce08-4dd1-abbe-e69e69ec7178 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:21.558797 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b0aea-ce08-4dd1-abbe-e69e69ec7178 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:21.560305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b0aea-ce08-4dd1-abbe-e69e69ec7178 service glance] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:21.570005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1150/2322] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:22.756238 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b352a-2449-4992-8256-bd96004e2c53 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:22.757746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5a5b352a-2449-4992-8256-bd96004e2c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:22.774383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:22.843264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b352a-2449-4992-8256-bd96004e2c53 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:22.853429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:22.854662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:22.856231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:22.856880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] PATH_INFO: `/v3/users/f870d738fb9346be841477fe9af19815` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:22.858836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f870d738fb9346be841477fe9af19815)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:22.860761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:23.090657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f870d738fb9346be841477fe9af19815'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:23.091395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f870d738fb9346be841477fe9af19815'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:23.099007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f870d738fb9346be841477fe9af19815'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:23.099699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f870d738fb9346be841477fe9af19815'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:23.118334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f870d738fb9346be841477fe9af19815'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:23.220069 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1173/2323] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:58:22 2022] DELETE /identity/v3/users/f870d738fb9346be841477fe9af19815 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:58:23.224878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2e32b-68dd-4656-acb8-48594e036d4f None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:23.225440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2e32b-68dd-4656-acb8-48594e036d4f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:23.225615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2e32b-68dd-4656-acb8-48594e036d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:23.246255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-52c2e32b-68dd-4656-acb8-48594e036d4f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:23.247991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-52c2e32b-68dd-4656-acb8-48594e036d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:23.324201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1151/2324] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:23.337017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:23.340357 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:23.455954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5742a-3e65-46af-a59e-4251ab3f962a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:23.462259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dbf1d-7f8d-427e-81cb-e681177656fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:23.463529 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbf1d-7f8d-427e-81cb-e681177656fe service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:23.464681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbf1d-7f8d-427e-81cb-e681177656fe service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:23.465116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dbf1d-7f8d-427e-81cb-e681177656fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:23.485510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700dbf1d-7f8d-427e-81cb-e681177656fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:23.487472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700dbf1d-7f8d-427e-81cb-e681177656fe service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:23.501382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1174/2325] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:58:24.176682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2e32b-68dd-4656-acb8-48594e036d4f None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:24.178139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-52c2e32b-68dd-4656-acb8-48594e036d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:24.191462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c2e32b-68dd-4656-acb8-48594e036d4f None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:24.197648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:24.199209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:24.199495 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:24.199765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] PATH_INFO: `/v3/projects/8d48a2ba51f24ce1acca365be720aa68` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:24.201001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d48a2ba51f24ce1acca365be720aa68)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:24.203110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:24.332857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d48a2ba51f24ce1acca365be720aa68', 'request_id': 'req-a90690d8-eb55-445d-b872-e4d3196b971c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:24.397010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d48a2ba51f24ce1acca365be720aa68', 'request_id': 'req-a90690d8-eb55-445d-b872-e4d3196b971c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:24.404064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d48a2ba51f24ce1acca365be720aa68', 'request_id': 'req-a90690d8-eb55-445d-b872-e4d3196b971c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:24.429583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] The token cache is being invalidate because project 8d48a2ba51f24ce1acca365be720aa68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:58:24.429952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:24.430808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:24.432500 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1152/2326] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:58:24 2022] DELETE /identity/v3/projects/8d48a2ba51f24ce1acca365be720aa68 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:24.437289 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dbf1d-7f8d-427e-81cb-e681177656fe service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:24.447167 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dbf1d-7f8d-427e-81cb-e681177656fe service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:24.482808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:24.516156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f36b1-258f-4e92-821b-d6a11bd02cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:24.517260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f36b1-258f-4e92-821b-d6a11bd02cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:24.517545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f36b1-258f-4e92-821b-d6a11bd02cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:24.517820 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f36b1-258f-4e92-821b-d6a11bd02cb2 admin admin] PATH_INFO: `/v3/users/ad833e56a1d84d50810a43ecbba2b858` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:24.519003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f36b1-258f-4e92-821b-d6a11bd02cb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad833e56a1d84d50810a43ecbba2b858)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:24.520990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f36b1-258f-4e92-821b-d6a11bd02cb2 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:24.746240 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-958f36b1-258f-4e92-821b-d6a11bd02cb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad833e56a1d84d50810a43ecbba2b858'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:24.747080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-958f36b1-258f-4e92-821b-d6a11bd02cb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad833e56a1d84d50810a43ecbba2b858'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:24.756567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-958f36b1-258f-4e92-821b-d6a11bd02cb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad833e56a1d84d50810a43ecbba2b858'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:24.762567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-958f36b1-258f-4e92-821b-d6a11bd02cb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad833e56a1d84d50810a43ecbba2b858'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:24.780140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-958f36b1-258f-4e92-821b-d6a11bd02cb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad833e56a1d84d50810a43ecbba2b858'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:24.866600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1175/2327] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:58:24 2022] DELETE /identity/v3/users/ad833e56a1d84d50810a43ecbba2b858 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:25.307373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:25.308904 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:25.325620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90690d8-eb55-445d-b872-e4d3196b971c admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:25.397429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:25.398311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:25.398597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:25.398874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] PATH_INFO: `/v3/projects/6503ce596c8841f98876a2a1c5a789e3` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:25.400076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6503ce596c8841f98876a2a1c5a789e3)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:25.402634 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:25.404631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:25.958128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6503ce596c8841f98876a2a1c5a789e3', 'request_id': 'req-d304a4ad-1935-4b4b-9fda-ad23629218c1'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:26.048791 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6503ce596c8841f98876a2a1c5a789e3', 'request_id': 'req-d304a4ad-1935-4b4b-9fda-ad23629218c1'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:26.055452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6503ce596c8841f98876a2a1c5a789e3', 'request_id': 'req-d304a4ad-1935-4b4b-9fda-ad23629218c1'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:26.072545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] The token cache is being invalidate because project 6503ce596c8841f98876a2a1c5a789e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:58:26.072930 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:26.073868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d304a4ad-1935-4b4b-9fda-ad23629218c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:26.079987 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1153/2328] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:58:25 2022] DELETE /identity/v3/projects/6503ce596c8841f98876a2a1c5a789e3 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:26.082592 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d5f8b-7027-41b9-bbe7-ff53b02cc7c0 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:26.087344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d5f8b-7027-41b9-bbe7-ff53b02cc7c0 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:26.087344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d5f8b-7027-41b9-bbe7-ff53b02cc7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:26.107801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ba0d5f8b-7027-41b9-bbe7-ff53b02cc7c0 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:26.109090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ba0d5f8b-7027-41b9-bbe7-ff53b02cc7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:26.179899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1176/2329] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:26.189259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bcf59-f589-4244-aafe-d612518b7b0d None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:26.196180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bcf59-f589-4244-aafe-d612518b7b0d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:26.196180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141bcf59-f589-4244-aafe-d612518b7b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:26.209478 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-141bcf59-f589-4244-aafe-d612518b7b0d None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:26.210565 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-141bcf59-f589-4244-aafe-d612518b7b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:26.219697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1154/2330] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:26.233464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d5f8b-7027-41b9-bbe7-ff53b02cc7c0 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:26.237301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d5f8b-7027-41b9-bbe7-ff53b02cc7c0 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:26.242318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30515931-d4d6-47e3-b587-449031d7e598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:26.242978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30515931-d4d6-47e3-b587-449031d7e598 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:26.243263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30515931-d4d6-47e3-b587-449031d7e598 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:26.243538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30515931-d4d6-47e3-b587-449031d7e598 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:26.244259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30515931-d4d6-47e3-b587-449031d7e598 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:58:26.244565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30515931-d4d6-47e3-b587-449031d7e598 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:26.246193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30515931-d4d6-47e3-b587-449031d7e598 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:26.258641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1177/2331] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:58:26.263775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bcf59-f589-4244-aafe-d612518b7b0d None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:26.273309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141bcf59-f589-4244-aafe-d612518b7b0d None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:26.278209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539717c-0085-4c16-add7-e7c0b80a87b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:26.278864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539717c-0085-4c16-add7-e7c0b80a87b6 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:26.279151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539717c-0085-4c16-add7-e7c0b80a87b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:26.279505 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539717c-0085-4c16-add7-e7c0b80a87b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:26.280175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539717c-0085-4c16-add7-e7c0b80a87b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:26.282127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539717c-0085-4c16-add7-e7c0b80a87b6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:26.354943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1155/2332] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:58:26 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:26.359429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30515931-d4d6-47e3-b587-449031d7e598 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:26.369902 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30515931-d4d6-47e3-b587-449031d7e598 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:26.403020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:26.426884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2f7d2-20f9-4192-8f83-38d5b5611d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:26.427545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2f7d2-20f9-4192-8f83-38d5b5611d18 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:26.427825 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2f7d2-20f9-4192-8f83-38d5b5611d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:26.428097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2f7d2-20f9-4192-8f83-38d5b5611d18 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:26.428773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2f7d2-20f9-4192-8f83-38d5b5611d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:26.430715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2f7d2-20f9-4192-8f83-38d5b5611d18 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:26.529068 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1178/2333] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:58:26 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:26.538352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539717c-0085-4c16-add7-e7c0b80a87b6 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:26.548907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539717c-0085-4c16-add7-e7c0b80a87b6 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:26.553780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31430726-d91a-4b14-a763-5438a51375a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:26.554667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430726-d91a-4b14-a763-5438a51375a8 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:26.555110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430726-d91a-4b14-a763-5438a51375a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:26.555551 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31430726-d91a-4b14-a763-5438a51375a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:26.556459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31430726-d91a-4b14-a763-5438a51375a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:26.558279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31430726-d91a-4b14-a763-5438a51375a8 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:26.573073 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1156/2334] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:26.574449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2f7d2-20f9-4192-8f83-38d5b5611d18 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:26.585974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2f7d2-20f9-4192-8f83-38d5b5611d18 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:26.590708 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b033-257f-4e8f-8ff6-ee797a3d989c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:26.591532 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b033-257f-4e8f-8ff6-ee797a3d989c admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:26.591813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b033-257f-4e8f-8ff6-ee797a3d989c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:26.592081 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b033-257f-4e8f-8ff6-ee797a3d989c admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:26.592823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b033-257f-4e8f-8ff6-ee797a3d989c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:26.594452 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b033-257f-4e8f-8ff6-ee797a3d989c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:26.604631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1179/2335] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:26.609866 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31430726-d91a-4b14-a763-5438a51375a8 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:26.619841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31430726-d91a-4b14-a763-5438a51375a8 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:26.628175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32453583-75da-41b9-a868-eb9865bd9eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:26.629083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32453583-75da-41b9-a868-eb9865bd9eae admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:26.629440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32453583-75da-41b9-a868-eb9865bd9eae admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:26.629743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32453583-75da-41b9-a868-eb9865bd9eae admin admin] PATH_INFO: `/v3/projects/e51ca85c03f745449d3d806bd2a1afcb/users/9ed705266f394500be5ebbe7020de47f/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:26.645189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32453583-75da-41b9-a868-eb9865bd9eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e51ca85c03f745449d3d806bd2a1afcb, user_id=9ed705266f394500be5ebbe7020de47f, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:26.647933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32453583-75da-41b9-a868-eb9865bd9eae admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:26.706246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1157/2336] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:26 2022] PUT /identity/v3/projects/e51ca85c03f745449d3d806bd2a1afcb/users/9ed705266f394500be5ebbe7020de47f/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:26.723416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1647-42b5-4fa5-8f42-a0a11db34e50 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:26.723711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1647-42b5-4fa5-8f42-a0a11db34e50 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:26.723989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1647-42b5-4fa5-8f42-a0a11db34e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:26.751824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ec2b1647-42b5-4fa5-8f42-a0a11db34e50 None None] MFA Rules not processed for user `9ed705266f394500be5ebbe7020de47f`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:26.752939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ec2b1647-42b5-4fa5-8f42-a0a11db34e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:26.820665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1180/2337] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:26.828049 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32453583-75da-41b9-a868-eb9865bd9eae admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:26.889599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32453583-75da-41b9-a868-eb9865bd9eae admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:26.894399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ed1d9-4044-43b0-aa38-5549de585af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:26.895054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ed1d9-4044-43b0-aa38-5549de585af9 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:26.895343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ed1d9-4044-43b0-aa38-5549de585af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:26.895619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ed1d9-4044-43b0-aa38-5549de585af9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:26.896291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ed1d9-4044-43b0-aa38-5549de585af9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:26.898442 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ed1d9-4044-43b0-aa38-5549de585af9 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:26.979437 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1158/2338] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:58:26 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:26.980910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b1647-42b5-4fa5-8f42-a0a11db34e50 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:26.992104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b1647-42b5-4fa5-8f42-a0a11db34e50 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:27.046849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1d9eb-1272-40dc-aadd-aadc952a61cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:27.047432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1d9eb-1272-40dc-aadd-aadc952a61cd admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:27.047717 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1d9eb-1272-40dc-aadd-aadc952a61cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:27.047991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1d9eb-1272-40dc-aadd-aadc952a61cd admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:27.048671 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1d9eb-1272-40dc-aadd-aadc952a61cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:27.050623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1d9eb-1272-40dc-aadd-aadc952a61cd admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:27.146336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1181/2339] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:58:26 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:27.150297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ed1d9-4044-43b0-aa38-5549de585af9 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:27.161786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ed1d9-4044-43b0-aa38-5549de585af9 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:27.166718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb81b4e-558a-4235-bb75-bcf91cccc124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:27.167393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb81b4e-558a-4235-bb75-bcf91cccc124 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:27.167699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb81b4e-558a-4235-bb75-bcf91cccc124 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:27.167951 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb81b4e-558a-4235-bb75-bcf91cccc124 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:27.168692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb81b4e-558a-4235-bb75-bcf91cccc124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:27.170910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb81b4e-558a-4235-bb75-bcf91cccc124 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:27.176986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:27.183964 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1159/2340] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:27.187273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1d9eb-1272-40dc-aadd-aadc952a61cd admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:27.199049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1d9eb-1272-40dc-aadd-aadc952a61cd admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:27.204403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be62dbf9-b9c7-4279-8eb9-64b48c1893ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:27.205198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62dbf9-b9c7-4279-8eb9-64b48c1893ec admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:27.205602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62dbf9-b9c7-4279-8eb9-64b48c1893ec admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:27.205951 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be62dbf9-b9c7-4279-8eb9-64b48c1893ec admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:27.208005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62dbf9-b9c7-4279-8eb9-64b48c1893ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:27.208720 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be62dbf9-b9c7-4279-8eb9-64b48c1893ec admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:27.221278 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1182/2341] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:27.224936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb81b4e-558a-4235-bb75-bcf91cccc124 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:27.235932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb81b4e-558a-4235-bb75-bcf91cccc124 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:27.240666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f707ad-8e44-49a4-90e8-41ce138b0174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:27.241630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f707ad-8e44-49a4-90e8-41ce138b0174 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:27.241920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f707ad-8e44-49a4-90e8-41ce138b0174 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:27.242228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f707ad-8e44-49a4-90e8-41ce138b0174 admin admin] PATH_INFO: `/v3/projects/439d6f596b0f48f787faf8848959e8f3/users/dba621ce3414450c8e6d2fd233fb3c07/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:27.257915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f707ad-8e44-49a4-90e8-41ce138b0174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=439d6f596b0f48f787faf8848959e8f3, user_id=dba621ce3414450c8e6d2fd233fb3c07, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:27.260803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f707ad-8e44-49a4-90e8-41ce138b0174 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:27.324192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1160/2342] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:27 2022] PUT /identity/v3/projects/439d6f596b0f48f787faf8848959e8f3/users/dba621ce3414450c8e6d2fd233fb3c07/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:27.335335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd3de8-fd87-4a88-91cd-d0ff22e6eb36 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:27.335627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd3de8-fd87-4a88-91cd-d0ff22e6eb36 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:27.335910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd3de8-fd87-4a88-91cd-d0ff22e6eb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:27.372164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-8dfd3de8-fd87-4a88-91cd-d0ff22e6eb36 None None] MFA Rules not processed for user `dba621ce3414450c8e6d2fd233fb3c07`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:27.372164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8dfd3de8-fd87-4a88-91cd-d0ff22e6eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:27.382361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:27.445196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1183/2343] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:27.454277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f707ad-8e44-49a4-90e8-41ce138b0174 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:27.526165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f707ad-8e44-49a4-90e8-41ce138b0174 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:27.531623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677eb8d6-4413-4f35-890f-ca86ffcbe4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:27.532375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677eb8d6-4413-4f35-890f-ca86ffcbe4b9 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:27.532663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677eb8d6-4413-4f35-890f-ca86ffcbe4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:27.532944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677eb8d6-4413-4f35-890f-ca86ffcbe4b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:27.533690 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677eb8d6-4413-4f35-890f-ca86ffcbe4b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:27.535605 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677eb8d6-4413-4f35-890f-ca86ffcbe4b9 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:27.650385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1161/2344] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:58:27 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:27.655168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd3de8-fd87-4a88-91cd-d0ff22e6eb36 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:27.666347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd3de8-fd87-4a88-91cd-d0ff22e6eb36 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:27.736063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e0c1d-e6a1-4b1e-860c-d134b696cbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:27.736845 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e0c1d-e6a1-4b1e-860c-d134b696cbea admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:27.737210 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e0c1d-e6a1-4b1e-860c-d134b696cbea admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:27.737580 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e0c1d-e6a1-4b1e-860c-d134b696cbea admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:27.738341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e0c1d-e6a1-4b1e-860c-d134b696cbea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:27.740387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e0c1d-e6a1-4b1e-860c-d134b696cbea admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:27.857042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1184/2345] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:58:27 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 18:58:27.860023 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677eb8d6-4413-4f35-890f-ca86ffcbe4b9 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:27.870738 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677eb8d6-4413-4f35-890f-ca86ffcbe4b9 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:27.886608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac01fb-f2c8-43b6-8274-927c415dd61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:27.887340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac01fb-f2c8-43b6-8274-927c415dd61a admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:27.888184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac01fb-f2c8-43b6-8274-927c415dd61a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:27.888469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac01fb-f2c8-43b6-8274-927c415dd61a admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:27.889204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac01fb-f2c8-43b6-8274-927c415dd61a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:27.891254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac01fb-f2c8-43b6-8274-927c415dd61a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:27.905540 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1162/2346] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:27.910439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e0c1d-e6a1-4b1e-860c-d134b696cbea admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:27.927629 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e0c1d-e6a1-4b1e-860c-d134b696cbea admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:27.932239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0720e-89f3-4615-ab45-a87ac6ae7765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:27.933316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0720e-89f3-4615-ab45-a87ac6ae7765 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:27.933619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0720e-89f3-4615-ab45-a87ac6ae7765 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:27.933899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0720e-89f3-4615-ab45-a87ac6ae7765 admin admin] PATH_INFO: `/v3/projects/d11c495ff15345e684001ef9a8c0b203/users/eaa269d30f61486e8eed4d695e1bacca/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:27.949314 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0720e-89f3-4615-ab45-a87ac6ae7765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11c495ff15345e684001ef9a8c0b203, user_id=eaa269d30f61486e8eed4d695e1bacca, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:27.952619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0720e-89f3-4615-ab45-a87ac6ae7765 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:28.035322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1185/2347] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:27 2022] PUT /identity/v3/projects/d11c495ff15345e684001ef9a8c0b203/users/eaa269d30f61486e8eed4d695e1bacca/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:28.039441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac01fb-f2c8-43b6-8274-927c415dd61a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:28.049961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac01fb-f2c8-43b6-8274-927c415dd61a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:28.114142 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1eb1c-c8ad-4121-b2cc-56a731dbaf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:28.114825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1eb1c-c8ad-4121-b2cc-56a731dbaf74 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:28.115109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1eb1c-c8ad-4121-b2cc-56a731dbaf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:28.115406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1eb1c-c8ad-4121-b2cc-56a731dbaf74 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:28.116138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1eb1c-c8ad-4121-b2cc-56a731dbaf74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:28.117945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1eb1c-c8ad-4121-b2cc-56a731dbaf74 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:28.132260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1163/2348] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:28.136799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0720e-89f3-4615-ab45-a87ac6ae7765 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:28.146672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0720e-89f3-4615-ab45-a87ac6ae7765 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:28.151453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b679a-121c-44dd-8177-67f4e3486bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:28.152331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b679a-121c-44dd-8177-67f4e3486bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:28.152617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b679a-121c-44dd-8177-67f4e3486bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:28.152896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b679a-121c-44dd-8177-67f4e3486bd1 admin admin] PATH_INFO: `/v3/projects/d11c495ff15345e684001ef9a8c0b203/users/eaa269d30f61486e8eed4d695e1bacca/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:28.154906 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b679a-121c-44dd-8177-67f4e3486bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11c495ff15345e684001ef9a8c0b203, user_id=eaa269d30f61486e8eed4d695e1bacca, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:28.157578 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b679a-121c-44dd-8177-67f4e3486bd1 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:28.168009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1186/2349] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:28 2022] PUT /identity/v3/projects/d11c495ff15345e684001ef9a8c0b203/users/eaa269d30f61486e8eed4d695e1bacca/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:28.179682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ddce4-c171-47ad-91f9-aa46358d3766 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:28.179999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ddce4-c171-47ad-91f9-aa46358d3766 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:28.180283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ddce4-c171-47ad-91f9-aa46358d3766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:28.207584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2c9ddce4-c171-47ad-91f9-aa46358d3766 None None] MFA Rules not processed for user `eaa269d30f61486e8eed4d695e1bacca`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:28.208805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2c9ddce4-c171-47ad-91f9-aa46358d3766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:28.245771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:28.285559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1164/2350] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:28.296942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b679a-121c-44dd-8177-67f4e3486bd1 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:28.298683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-eb0b679a-121c-44dd-8177-67f4e3486bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:28.369016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b679a-121c-44dd-8177-67f4e3486bd1 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:28.370207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694c958-a02b-4ec3-9119-c7abdc3aac3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:28.370882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694c958-a02b-4ec3-9119-c7abdc3aac3c service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:28.371174 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694c958-a02b-4ec3-9119-c7abdc3aac3c service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:28.371452 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694c958-a02b-4ec3-9119-c7abdc3aac3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:28.392294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:28.433850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694c958-a02b-4ec3-9119-c7abdc3aac3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:28.435691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4694c958-a02b-4ec3-9119-c7abdc3aac3c service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:28.444810 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1187/2351] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:30.562034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ddce4-c171-47ad-91f9-aa46358d3766 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:30.571350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ddce4-c171-47ad-91f9-aa46358d3766 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:30.575430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9429820-cdaa-4314-8dd6-82b7862b2cf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:30.576190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9429820-cdaa-4314-8dd6-82b7862b2cf8 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:30.576341 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9429820-cdaa-4314-8dd6-82b7862b2cf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:30.576620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9429820-cdaa-4314-8dd6-82b7862b2cf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:30.641773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9429820-cdaa-4314-8dd6-82b7862b2cf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:30.643242 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9429820-cdaa-4314-8dd6-82b7862b2cf8 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:30.652328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1165/2352] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:32.096385 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694c958-a02b-4ec3-9119-c7abdc3aac3c service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:32.100863 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4694c958-a02b-4ec3-9119-c7abdc3aac3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:32.111443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694c958-a02b-4ec3-9119-c7abdc3aac3c service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:32.116000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada716f-e7ca-4548-bc76-054d914cf994 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:32.116725 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada716f-e7ca-4548-bc76-054d914cf994 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:32.117264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada716f-e7ca-4548-bc76-054d914cf994 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:32.117548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada716f-e7ca-4548-bc76-054d914cf994 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:32.127386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada716f-e7ca-4548-bc76-054d914cf994 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:32.128812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada716f-e7ca-4548-bc76-054d914cf994 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:32.139266 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1188/2353] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:36.307456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9429820-cdaa-4314-8dd6-82b7862b2cf8 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:36.308961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c9429820-cdaa-4314-8dd6-82b7862b2cf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:36.375054 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9429820-cdaa-4314-8dd6-82b7862b2cf8 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:36.378779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ce85d-e1f2-4bab-9d65-3292fc46dbe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:36.379433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce85d-e1f2-4bab-9d65-3292fc46dbe5 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:36.379716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce85d-e1f2-4bab-9d65-3292fc46dbe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:36.380022 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ce85d-e1f2-4bab-9d65-3292fc46dbe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:36.381956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5c2ce85d-e1f2-4bab-9d65-3292fc46dbe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:36.453169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ce85d-e1f2-4bab-9d65-3292fc46dbe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:36.454966 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ce85d-e1f2-4bab-9d65-3292fc46dbe5 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:36.467349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1166/2354] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:36.642038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada716f-e7ca-4548-bc76-054d914cf994 service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:36.715386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada716f-e7ca-4548-bc76-054d914cf994 service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:36.724143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:36.725764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:36.726609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:36.727025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] PATH_INFO: `/v3/users/9ed705266f394500be5ebbe7020de47f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:36.729350 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ed705266f394500be5ebbe7020de47f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:36.732929 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:36.920658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ed705266f394500be5ebbe7020de47f'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:36.921382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ed705266f394500be5ebbe7020de47f'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:36.928718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ed705266f394500be5ebbe7020de47f'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:36.929485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ed705266f394500be5ebbe7020de47f'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:36.942669 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ed705266f394500be5ebbe7020de47f'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:37.004448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1189/2355] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:58:36 2022] DELETE /identity/v3/users/9ed705266f394500be5ebbe7020de47f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:37.009157 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f7d1-7c17-472f-832c-e92e75bb936c None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:37.009561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f7d1-7c17-472f-832c-e92e75bb936c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:37.009914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f7d1-7c17-472f-832c-e92e75bb936c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:37.029042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-9458f7d1-7c17-472f-832c-e92e75bb936c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:37.030433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-9458f7d1-7c17-472f-832c-e92e75bb936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:37.044154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:37.104952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1167/2356] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:37.117113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:37.119862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:37.151571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:37.181638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f7a2b-057f-4b9a-8d99-22cf0b20039f admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:37.186668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe4a45-db83-4a2e-99e9-86f9e75c5b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:37.187327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe4a45-db83-4a2e-99e9-86f9e75c5b1b service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:37.187611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe4a45-db83-4a2e-99e9-86f9e75c5b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:37.187886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe4a45-db83-4a2e-99e9-86f9e75c5b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:37.198197 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe4a45-db83-4a2e-99e9-86f9e75c5b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:37.199666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe4a45-db83-4a2e-99e9-86f9e75c5b1b service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:37.223005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1190/2357] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:37.656654 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458f7d1-7c17-472f-832c-e92e75bb936c None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:37.658201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-9458f7d1-7c17-472f-832c-e92e75bb936c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:37.674946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458f7d1-7c17-472f-832c-e92e75bb936c None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:37.683911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:37.685937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:37.686649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:37.687002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] PATH_INFO: `/v3/projects/e51ca85c03f745449d3d806bd2a1afcb` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:37.688742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e51ca85c03f745449d3d806bd2a1afcb)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:37.691156 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:37.809213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e51ca85c03f745449d3d806bd2a1afcb', 'request_id': 'req-79be8bdf-469c-4539-addf-0271c9dcc533'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:37.871874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e51ca85c03f745449d3d806bd2a1afcb', 'request_id': 'req-79be8bdf-469c-4539-addf-0271c9dcc533'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:37.877700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e51ca85c03f745449d3d806bd2a1afcb', 'request_id': 'req-79be8bdf-469c-4539-addf-0271c9dcc533'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:37.889294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] The token cache is being invalidate because project e51ca85c03f745449d3d806bd2a1afcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:58:37.889586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:37.890229 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:37.891756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1168/2358] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:58:37 2022] DELETE /identity/v3/projects/e51ca85c03f745449d3d806bd2a1afcb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:37.895732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe4a45-db83-4a2e-99e9-86f9e75c5b1b service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:37.906113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe4a45-db83-4a2e-99e9-86f9e75c5b1b service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:37.967944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:37.968851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:37.969150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:37.969472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] PATH_INFO: `/v3/users/dba621ce3414450c8e6d2fd233fb3c07` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:37.970738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dba621ce3414450c8e6d2fd233fb3c07)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:37.972722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:38.188494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dba621ce3414450c8e6d2fd233fb3c07'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:38.189060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dba621ce3414450c8e6d2fd233fb3c07'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:38.196510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dba621ce3414450c8e6d2fd233fb3c07'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:38.197155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dba621ce3414450c8e6d2fd233fb3c07'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:38.213767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dba621ce3414450c8e6d2fd233fb3c07'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:38.306268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1191/2359] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:58:37 2022] DELETE /identity/v3/users/dba621ce3414450c8e6d2fd233fb3c07 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 18:58:39.477806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:39.479832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:39.510870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79be8bdf-469c-4539-addf-0271c9dcc533 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:39.629655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:39.631196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:39.631583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:39.632268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] PATH_INFO: `/v3/projects/439d6f596b0f48f787faf8848959e8f3` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:39.634337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=439d6f596b0f48f787faf8848959e8f3)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:39.636397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:39.772274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:39.782047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '439d6f596b0f48f787faf8848959e8f3', 'request_id': 'req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:39.866073 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '439d6f596b0f48f787faf8848959e8f3', 'request_id': 'req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:39.879823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '439d6f596b0f48f787faf8848959e8f3', 'request_id': 'req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:39.902342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] The token cache is being invalidate because project 439d6f596b0f48f787faf8848959e8f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:58:39.902633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:39.904499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:39.906933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1169/2360] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:58:39 2022] DELETE /identity/v3/projects/439d6f596b0f48f787faf8848959e8f3 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:39.910750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:39.912300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:39.935802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b25da7-7fe4-476c-b56e-75b15734be8f admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:40.019116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:40.049193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4abffd-2706-4d50-87f7-d98bbc4b1edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:40.054464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4abffd-2706-4d50-87f7-d98bbc4b1edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:40.054464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4abffd-2706-4d50-87f7-d98bbc4b1edf admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:40.054464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4abffd-2706-4d50-87f7-d98bbc4b1edf admin admin] PATH_INFO: `/v3/users/eaa269d30f61486e8eed4d695e1bacca` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:40.054464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4abffd-2706-4d50-87f7-d98bbc4b1edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa269d30f61486e8eed4d695e1bacca)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:40.054819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4abffd-2706-4d50-87f7-d98bbc4b1edf admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:40.231007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cd4abffd-2706-4d50-87f7-d98bbc4b1edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa269d30f61486e8eed4d695e1bacca'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:40.232549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cd4abffd-2706-4d50-87f7-d98bbc4b1edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa269d30f61486e8eed4d695e1bacca'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:40.249308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cd4abffd-2706-4d50-87f7-d98bbc4b1edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa269d30f61486e8eed4d695e1bacca'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:40.251520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cd4abffd-2706-4d50-87f7-d98bbc4b1edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa269d30f61486e8eed4d695e1bacca'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:40.275011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cd4abffd-2706-4d50-87f7-d98bbc4b1edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa269d30f61486e8eed4d695e1bacca'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:40.335032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1192/2361] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:58:39 2022] DELETE /identity/v3/users/eaa269d30f61486e8eed4d695e1bacca => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:41.230077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:41.231616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:41.248692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c12c9-a83f-4089-b8c9-61b7624f9cc6 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:41.308255 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:41.310183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:41.310763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:41.311252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] PATH_INFO: `/v3/projects/d11c495ff15345e684001ef9a8c0b203` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:41.312703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11c495ff15345e684001ef9a8c0b203)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:41.314930 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:41.351024 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593444e-4281-426e-a840-2e150ed53426 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:41.351557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593444e-4281-426e-a840-2e150ed53426 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:41.352025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593444e-4281-426e-a840-2e150ed53426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:41.387341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0593444e-4281-426e-a840-2e150ed53426 None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:41.389958 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0593444e-4281-426e-a840-2e150ed53426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:41.426491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11c495ff15345e684001ef9a8c0b203', 'request_id': 'req-564aa462-d45b-4700-bfd1-ac451c848162'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:41.485299 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1193/2362] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 18:58:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 18:58:41.489012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11c495ff15345e684001ef9a8c0b203', 'request_id': 'req-564aa462-d45b-4700-bfd1-ac451c848162'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:41.496172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11c495ff15345e684001ef9a8c0b203', 'request_id': 'req-564aa462-d45b-4700-bfd1-ac451c848162'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:41.519788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] The token cache is being invalidate because project d11c495ff15345e684001ef9a8c0b203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:58:41.520079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:41.520742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:58:41.523298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1170/2363] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:58:41 2022] DELETE /identity/v3/projects/d11c495ff15345e684001ef9a8c0b203 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:41.527953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38882b69-41a5-4547-931e-6e5eb29959ea None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:41.528275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38882b69-41a5-4547-931e-6e5eb29959ea None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:41.528603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38882b69-41a5-4547-931e-6e5eb29959ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:41.546096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:41.547618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:41.569133 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-38882b69-41a5-4547-931e-6e5eb29959ea None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:41.570547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-38882b69-41a5-4547-931e-6e5eb29959ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:41.642027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aa462-d45b-4700-bfd1-ac451c848162 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:41.646130 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:41.648992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece0207-9259-4dca-a197-a20cdc34101f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:41.650099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece0207-9259-4dca-a197-a20cdc34101f service swift] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:41.650611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece0207-9259-4dca-a197-a20cdc34101f service swift] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:41.651029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece0207-9259-4dca-a197-a20cdc34101f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:41.653983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0ece0207-9259-4dca-a197-a20cdc34101f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:41.660574 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:41.711613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1194/2364] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:41.722611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c587e-f82c-46f6-814a-f2d3bbf30835 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:41.722909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c587e-f82c-46f6-814a-f2d3bbf30835 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:41.723193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c587e-f82c-46f6-814a-f2d3bbf30835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:41.758625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-dd8c587e-f82c-46f6-814a-f2d3bbf30835 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:41.761077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece0207-9259-4dca-a197-a20cdc34101f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:41.761772 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-dd8c587e-f82c-46f6-814a-f2d3bbf30835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:41.763733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece0207-9259-4dca-a197-a20cdc34101f service swift] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:41.778242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1195/2365] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:41.779750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c587e-f82c-46f6-814a-f2d3bbf30835 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:41.785599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1171/2366] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 18:58:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 18:58:41.793078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c587e-f82c-46f6-814a-f2d3bbf30835 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:41.810194 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10326a24-e2cb-4a2d-a55f-e21d1820a095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:41.810948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10326a24-e2cb-4a2d-a55f-e21d1820a095 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:41.811367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10326a24-e2cb-4a2d-a55f-e21d1820a095 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:41.811605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10326a24-e2cb-4a2d-a55f-e21d1820a095 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:41.812398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10326a24-e2cb-4a2d-a55f-e21d1820a095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 18:58:41.812866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10326a24-e2cb-4a2d-a55f-e21d1820a095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:41.814407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10326a24-e2cb-4a2d-a55f-e21d1820a095 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:41.844246 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1196/2367] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 18:58:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 18:58:41.845568 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece0207-9259-4dca-a197-a20cdc34101f service swift] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:41.862905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece0207-9259-4dca-a197-a20cdc34101f service swift] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:41.869116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f25be-be2c-4172-8b6e-70180b927bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:41.869850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f25be-be2c-4172-8b6e-70180b927bdc admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:41.870131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f25be-be2c-4172-8b6e-70180b927bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:41.870404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f25be-be2c-4172-8b6e-70180b927bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:41.871092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f25be-be2c-4172-8b6e-70180b927bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:41.873582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f25be-be2c-4172-8b6e-70180b927bdc admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:41.970315 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1172/2368] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 18:58:41 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:41.975233 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10326a24-e2cb-4a2d-a55f-e21d1820a095 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:41.990172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10326a24-e2cb-4a2d-a55f-e21d1820a095 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:42.088636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3487a8-7f59-4271-adf0-f2a3d0ab57b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:42.090187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3487a8-7f59-4271-adf0-f2a3d0ab57b8 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:42.090579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3487a8-7f59-4271-adf0-f2a3d0ab57b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:42.090891 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3487a8-7f59-4271-adf0-f2a3d0ab57b8 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:42.091601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3487a8-7f59-4271-adf0-f2a3d0ab57b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:42.093734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3487a8-7f59-4271-adf0-f2a3d0ab57b8 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:42.231208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1197/2369] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 18:58:41 2022] POST /identity/v3/users => generated 287 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 18:58:42.233515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f25be-be2c-4172-8b6e-70180b927bdc admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:42.250792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f25be-be2c-4172-8b6e-70180b927bdc admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:42.304589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86465d6-be75-4b54-b23f-610eeb9c6713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:42.305483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86465d6-be75-4b54-b23f-610eeb9c6713 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:42.305863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86465d6-be75-4b54-b23f-610eeb9c6713 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:42.306272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86465d6-be75-4b54-b23f-610eeb9c6713 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:42.307306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86465d6-be75-4b54-b23f-610eeb9c6713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:42.309001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86465d6-be75-4b54-b23f-610eeb9c6713 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:42.342295 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1173/2370] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:42.351029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3487a8-7f59-4271-adf0-f2a3d0ab57b8 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:42.362601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3487a8-7f59-4271-adf0-f2a3d0ab57b8 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:42.369715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730933ab-5236-4edf-979b-955e7b27c8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:42.378571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730933ab-5236-4edf-979b-955e7b27c8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:42.378571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730933ab-5236-4edf-979b-955e7b27c8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:42.378571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730933ab-5236-4edf-979b-955e7b27c8f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:42.378571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730933ab-5236-4edf-979b-955e7b27c8f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:42.378571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730933ab-5236-4edf-979b-955e7b27c8f3 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:42.386262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1198/2371] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 18:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 18:58:42.390984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86465d6-be75-4b54-b23f-610eeb9c6713 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:42.402874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86465d6-be75-4b54-b23f-610eeb9c6713 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:42.408268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330fb94-dc6c-4ba4-acf5-ffc9eaf28c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:42.409282 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330fb94-dc6c-4ba4-acf5-ffc9eaf28c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:42.409580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330fb94-dc6c-4ba4-acf5-ffc9eaf28c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:42.409860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330fb94-dc6c-4ba4-acf5-ffc9eaf28c17 admin admin] PATH_INFO: `/v3/projects/c5e5a9406f7f4ccba4aaf2cccf1de07e/users/325f963f409b4fd68f9ac9a29de4be72/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:42.439144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330fb94-dc6c-4ba4-acf5-ffc9eaf28c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e5a9406f7f4ccba4aaf2cccf1de07e, user_id=325f963f409b4fd68f9ac9a29de4be72, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:42.444937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330fb94-dc6c-4ba4-acf5-ffc9eaf28c17 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:42.614828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1174/2372] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 18:58:42 2022] PUT /identity/v3/projects/c5e5a9406f7f4ccba4aaf2cccf1de07e/users/325f963f409b4fd68f9ac9a29de4be72/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:58:42.618999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ac822-203a-4431-ba9f-27b02730372b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:42.623432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ac822-203a-4431-ba9f-27b02730372b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:42.623980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ac822-203a-4431-ba9f-27b02730372b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:42.650737 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-413ac822-203a-4431-ba9f-27b02730372b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:42.652170 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-413ac822-203a-4431-ba9f-27b02730372b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:42.748742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1199/2373] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:42.759430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330fb94-dc6c-4ba4-acf5-ffc9eaf28c17 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:42.761178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8330fb94-dc6c-4ba4-acf5-ffc9eaf28c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:42.861373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330fb94-dc6c-4ba4-acf5-ffc9eaf28c17 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:42.867683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed80608-2d1b-446f-ae4e-16629caa5a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:42.867683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed80608-2d1b-446f-ae4e-16629caa5a45 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:42.867683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed80608-2d1b-446f-ae4e-16629caa5a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:42.868124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed80608-2d1b-446f-ae4e-16629caa5a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:42.879208 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed80608-2d1b-446f-ae4e-16629caa5a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:42.880963 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed80608-2d1b-446f-ae4e-16629caa5a45 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:42.898658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1175/2374] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:44.536719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e76eb3-ae16-4751-92b5-1efe137dbc1f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:44.537271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e76eb3-ae16-4751-92b5-1efe137dbc1f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:44.537429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e76eb3-ae16-4751-92b5-1efe137dbc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:44.566570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f2e76eb3-ae16-4751-92b5-1efe137dbc1f None None] MFA Rules not processed for user `325f963f409b4fd68f9ac9a29de4be72`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:44.568117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f2e76eb3-ae16-4751-92b5-1efe137dbc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:44.605886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:44.646282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1200/2375] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:44.658903 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed80608-2d1b-446f-ae4e-16629caa5a45 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:58:44.660379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-bed80608-2d1b-446f-ae4e-16629caa5a45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:44.739870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed80608-2d1b-446f-ae4e-16629caa5a45 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:58:44.748975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb64d1a-a5d8-46fe-9ed9-6b3d318c1894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:58:44.750332 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb64d1a-a5d8-46fe-9ed9-6b3d318c1894 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:44.750833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb64d1a-a5d8-46fe-9ed9-6b3d318c1894 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:44.751865 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb64d1a-a5d8-46fe-9ed9-6b3d318c1894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:44.759178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:44.764847 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb64d1a-a5d8-46fe-9ed9-6b3d318c1894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:58:44.767699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb64d1a-a5d8-46fe-9ed9-6b3d318c1894 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:58:44.780180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1176/2376] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:58:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:58:46.589128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222d107-3b32-463a-a5ad-92c7ecf00cab None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:46.589859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222d107-3b32-463a-a5ad-92c7ecf00cab None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:46.589859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222d107-3b32-463a-a5ad-92c7ecf00cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:46.617651 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-5222d107-3b32-463a-a5ad-92c7ecf00cab None None] MFA Rules not processed for user `325f963f409b4fd68f9ac9a29de4be72`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:46.619277 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5222d107-3b32-463a-a5ad-92c7ecf00cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:46.627887 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1201/2377] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:47.419253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd6e69-4951-4460-9df3-efb7aaf8c23f None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:58:47.420482 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd6e69-4951-4460-9df3-efb7aaf8c23f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:58:47.420867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd6e69-4951-4460-9df3-efb7aaf8c23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:58:47.443680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-63cd6e69-4951-4460-9df3-efb7aaf8c23f None None] MFA Rules not processed for user `1c0e7e9f9d0f409a8820d8bd8dd239cf`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:58:47.445077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-63cd6e69-4951-4460-9df3-efb7aaf8c23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:58:47.519939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1177/2378] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:58:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:58:56.605695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=14 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:58:57.507038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=28 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:01.016677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222d107-3b32-463a-a5ad-92c7ecf00cab None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:01.018154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5222d107-3b32-463a-a5ad-92c7ecf00cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:01.128316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222d107-3b32-463a-a5ad-92c7ecf00cab None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:01.133404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da3303-3021-4528-a365-315fd3502605 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:01.134054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da3303-3021-4528-a365-315fd3502605 service placement] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:01.134341 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da3303-3021-4528-a365-315fd3502605 service placement] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:01.134616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da3303-3021-4528-a365-315fd3502605 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:01.136488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-90da3303-3021-4528-a365-315fd3502605 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:01.149321 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da3303-3021-4528-a365-315fd3502605 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:01.151274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90da3303-3021-4528-a365-315fd3502605 service placement] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:01.161511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1202/2379] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:04.291926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea791649-0a22-4739-92ea-4e6d94ac3d8e None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:04.292458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea791649-0a22-4739-92ea-4e6d94ac3d8e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:04.297431 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea791649-0a22-4739-92ea-4e6d94ac3d8e None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:04.297431 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1178/2380] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 18:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 18:59:04.300431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adacb71-2a1c-4178-94c9-5100f33424d9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:04.300894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adacb71-2a1c-4178-94c9-5100f33424d9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:04.301063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adacb71-2a1c-4178-94c9-5100f33424d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:04.351116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6adacb71-2a1c-4178-94c9-5100f33424d9 None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:04.352345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6adacb71-2a1c-4178-94c9-5100f33424d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:04.431732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1203/2381] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 18:59:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:04.450337 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea791649-0a22-4739-92ea-4e6d94ac3d8e None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:04.452104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ea791649-0a22-4739-92ea-4e6d94ac3d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:04.532772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea791649-0a22-4739-92ea-4e6d94ac3d8e None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:04.537076 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe322e5-6013-45b6-8a2d-cbd88e543847 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17b7a7128d64b2ba548add02d3ec264', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:04.537721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe322e5-6013-45b6-8a2d-cbd88e543847 service glance] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:04.538006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe322e5-6013-45b6-8a2d-cbd88e543847 service glance] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:04.538281 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe322e5-6013-45b6-8a2d-cbd88e543847 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:04.549105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe322e5-6013-45b6-8a2d-cbd88e543847 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:04.550563 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe322e5-6013-45b6-8a2d-cbd88e543847 service glance] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:04.561366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1179/2382] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:04.789458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc1577-2635-4617-8d51-529f52a7b836 None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:04.789705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc1577-2635-4617-8d51-529f52a7b836 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:04.790012 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc1577-2635-4617-8d51-529f52a7b836 None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:04.791428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1204/2383] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 18:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:59:04.796152 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393c7b5-39eb-4d0d-b759-aa3d0b85e51f None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:04.796463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393c7b5-39eb-4d0d-b759-aa3d0b85e51f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:04.796742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393c7b5-39eb-4d0d-b759-aa3d0b85e51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:04.815250 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-7393c7b5-39eb-4d0d-b759-aa3d0b85e51f None None] MFA Rules not processed for user `f4e05d75cd1142e18d7518b78edd88fa`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:04.819125 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-7393c7b5-39eb-4d0d-b759-aa3d0b85e51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:04.823004 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1180/2384] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 18:59:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:04.835157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc1577-2635-4617-8d51-529f52a7b836 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:04.847417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc1577-2635-4617-8d51-529f52a7b836 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:04.850892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329abc77-0e70-4e6c-a4a4-b7619a4ef20c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:04.851559 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329abc77-0e70-4e6c-a4a4-b7619a4ef20c service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:04.851841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329abc77-0e70-4e6c-a4a4-b7619a4ef20c service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:04.852117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329abc77-0e70-4e6c-a4a4-b7619a4ef20c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:04.859513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329abc77-0e70-4e6c-a4a4-b7619a4ef20c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:04.860939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329abc77-0e70-4e6c-a4a4-b7619a4ef20c service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:04.868585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1205/2385] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:04.948225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad2a23-5c7e-443f-8957-d6eb934c3805 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:04.948545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad2a23-5c7e-443f-8957-d6eb934c3805 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:04.948819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ad2a23-5c7e-443f-8957-d6eb934c3805 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:04.950197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1181/2386] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 18:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 18:59:04.955934 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bb86f-b340-4676-98b2-17b1d217401e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:04.956228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bb86f-b340-4676-98b2-17b1d217401e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:04.956503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bb86f-b340-4676-98b2-17b1d217401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:04.976645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-a39bb86f-b340-4676-98b2-17b1d217401e None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:04.977901 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a39bb86f-b340-4676-98b2-17b1d217401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:04.988542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1206/2387] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 18:59:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:05.000953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad2a23-5c7e-443f-8957-d6eb934c3805 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:05.012744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ad2a23-5c7e-443f-8957-d6eb934c3805 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:05.017011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e207-6d71-4818-886e-c88b88ffd7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:05.017697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e207-6d71-4818-886e-c88b88ffd7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:05.017983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e207-6d71-4818-886e-c88b88ffd7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:05.018287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e207-6d71-4818-886e-c88b88ffd7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:05.028499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5e207-6d71-4818-886e-c88b88ffd7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:05.030710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5e207-6d71-4818-886e-c88b88ffd7d1 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:05.041401 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1182/2388] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:10.290283 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bb86f-b340-4676-98b2-17b1d217401e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:10.291757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a39bb86f-b340-4676-98b2-17b1d217401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:10.310604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bb86f-b340-4676-98b2-17b1d217401e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:10.315316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:10.316193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.316495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.316753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] PATH_INFO: `/v3/users/91745b9228804ee1844aa498ad3d3ae9` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.318078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] RBAC: Authorizing `identity:delete_user(user_id=91745b9228804ee1844aa498ad3d3ae9)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:10.320020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:10.323462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:10.467955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91745b9228804ee1844aa498ad3d3ae9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:10.468634 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91745b9228804ee1844aa498ad3d3ae9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:10.476015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91745b9228804ee1844aa498ad3d3ae9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:10.481363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91745b9228804ee1844aa498ad3d3ae9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:10.489108 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91745b9228804ee1844aa498ad3d3ae9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:10.557884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1207/2389] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 18:59:10 2022] DELETE /identity/v3/users/91745b9228804ee1844aa498ad3d3ae9 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:10.562371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed042e-cfb0-4d2c-ae97-9fca96070f7e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.562882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed042e-cfb0-4d2c-ae97-9fca96070f7e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.562976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed042e-cfb0-4d2c-ae97-9fca96070f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.581432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-81ed042e-cfb0-4d2c-ae97-9fca96070f7e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:10.582664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-81ed042e-cfb0-4d2c-ae97-9fca96070f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:10.644044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1183/2390] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:10.653681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:10.655080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:10.711087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012821d-cc11-4abe-a003-e8c36eb2084f admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:10.714749 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28cb9a-9cd1-49ce-8719-e7e685ae3da1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:10.715397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28cb9a-9cd1-49ce-8719-e7e685ae3da1 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:10.715686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28cb9a-9cd1-49ce-8719-e7e685ae3da1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:10.715964 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28cb9a-9cd1-49ce-8719-e7e685ae3da1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:10.724910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28cb9a-9cd1-49ce-8719-e7e685ae3da1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:10.726373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c28cb9a-9cd1-49ce-8719-e7e685ae3da1 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:10.736552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1208/2391] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 18:59:11.136796 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed042e-cfb0-4d2c-ae97-9fca96070f7e None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:11.138301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-81ed042e-cfb0-4d2c-ae97-9fca96070f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:11.149936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed042e-cfb0-4d2c-ae97-9fca96070f7e None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:11.155042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:11.155967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:11.156251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:11.156522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] PATH_INFO: `/v3/projects/7969c68692884392ad5321b686df4801` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:11.158214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7969c68692884392ad5321b686df4801)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:11.160124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:11.244133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7969c68692884392ad5321b686df4801', 'request_id': 'req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:11.300901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7969c68692884392ad5321b686df4801', 'request_id': 'req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:11.302956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:11.306738 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7969c68692884392ad5321b686df4801', 'request_id': 'req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:11.317423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] The token cache is being invalidate because project 7969c68692884392ad5321b686df4801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 18:59:11.317718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:11.318376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b218358e-e1f5-4853-8ba6-54cee9ef8e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 18:59:11.320387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1184/2392] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 18:59:11 2022] DELETE /identity/v3/projects/7969c68692884392ad5321b686df4801 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 18:59:20.731443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:21.426984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=5,keystone:DELETE=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:25.346342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28cb9a-9cd1-49ce-8719-e7e685ae3da1 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:25.347801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-2c28cb9a-9cd1-49ce-8719-e7e685ae3da1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:25.359144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28cb9a-9cd1-49ce-8719-e7e685ae3da1 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:25.415871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38602b6-ccda-4414-9ce1-065b3fe6f419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:25.416516 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38602b6-ccda-4414-9ce1-065b3fe6f419 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:25.416796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38602b6-ccda-4414-9ce1-065b3fe6f419 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:25.417062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38602b6-ccda-4414-9ce1-065b3fe6f419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:25.418942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c38602b6-ccda-4414-9ce1-065b3fe6f419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:25.507041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38602b6-ccda-4414-9ce1-065b3fe6f419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:25.508472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38602b6-ccda-4414-9ce1-065b3fe6f419 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:25.521802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1209/2393] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 18:59:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 18:59:28.714971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400c1f0-d312-4890-97fe-92d785d4afa6 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:28.715506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400c1f0-d312-4890-97fe-92d785d4afa6 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:28.715614 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400c1f0-d312-4890-97fe-92d785d4afa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:28.734331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-8400c1f0-d312-4890-97fe-92d785d4afa6 None None] MFA Rules not processed for user `325f963f409b4fd68f9ac9a29de4be72`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:28.735818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8400c1f0-d312-4890-97fe-92d785d4afa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:28.791267 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1185/2394] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 18:59:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:32.635873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f5d2a-8331-4c53-add6-af5cb2455dda None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:32.636389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f5d2a-8331-4c53-add6-af5cb2455dda None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:32.636480 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f5d2a-8331-4c53-add6-af5cb2455dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:32.688686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4d5f5d2a-8331-4c53-add6-af5cb2455dda None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 18:59:32.690402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4d5f5d2a-8331-4c53-add6-af5cb2455dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:32.755704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1210/2395] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 18:59:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 18:59:32.767960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400c1f0-d312-4890-97fe-92d785d4afa6 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 18:59:32.769478 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8400c1f0-d312-4890-97fe-92d785d4afa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 18:59:32.877477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400c1f0-d312-4890-97fe-92d785d4afa6 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 18:59:32.882939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154a2b7-90cf-4ff2-957c-5b0e4b3bf042 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 18:59:32.883618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154a2b7-90cf-4ff2-957c-5b0e4b3bf042 service swift] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 18:59:32.883903 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154a2b7-90cf-4ff2-957c-5b0e4b3bf042 service swift] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 18:59:32.884178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154a2b7-90cf-4ff2-957c-5b0e4b3bf042 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 18:59:32.892839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154a2b7-90cf-4ff2-957c-5b0e4b3bf042 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 18:59:32.894548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154a2b7-90cf-4ff2-957c-5b0e4b3bf042 service swift] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 18:59:32.902942 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1186/2396] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 18:59:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 18:59:42.751920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=90 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 18:59:42.900002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=66 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:03.666570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a1053-89f6-4f17-b9c5-cbd9cf14c28b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.667283 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a1053-89f6-4f17-b9c5-cbd9cf14c28b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.667283 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a1053-89f6-4f17-b9c5-cbd9cf14c28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.685847 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-526a1053-89f6-4f17-b9c5-cbd9cf14c28b None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:03.687199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-526a1053-89f6-4f17-b9c5-cbd9cf14c28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:03.719338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1211/2397] 10.222.0.81 () {58 vars in 967 bytes} [Thu Jun 16 19:00:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:03.744802 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154a2b7-90cf-4ff2-957c-5b0e4b3bf042 service swift] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:03.755970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154a2b7-90cf-4ff2-957c-5b0e4b3bf042 service swift] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:03.761190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f7e3e-242c-48db-b824-492d8407bc5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:03.761890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f7e3e-242c-48db-b824-492d8407bc5b service swift] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:03.762170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f7e3e-242c-48db-b824-492d8407bc5b service swift] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:03.762442 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f7e3e-242c-48db-b824-492d8407bc5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:03.771668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f7e3e-242c-48db-b824-492d8407bc5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:03.773424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f7e3e-242c-48db-b824-492d8407bc5b service swift] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:03.781719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1187/2398] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:00:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:00:07.851399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372252b-313a-4899-ba45-cc8ae085a5b6 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:07.851863 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372252b-313a-4899-ba45-cc8ae085a5b6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:07.856600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372252b-313a-4899-ba45-cc8ae085a5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:07.870228 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d372252b-313a-4899-ba45-cc8ae085a5b6 None None] MFA Rules not processed for user `eed09fa6ecf14635949417f19c8a3af0`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:07.871449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d372252b-313a-4899-ba45-cc8ae085a5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:07.929657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1212/2399] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:00:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:13.401727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5dbcf-2480-4d18-b04a-6e0c6cfb072c None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:13.402226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5dbcf-2480-4d18-b04a-6e0c6cfb072c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:13.402335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5dbcf-2480-4d18-b04a-6e0c6cfb072c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:13.422671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-53a5dbcf-2480-4d18-b04a-6e0c6cfb072c None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:13.423879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-53a5dbcf-2480-4d18-b04a-6e0c6cfb072c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:13.431921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1188/2400] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 19:00:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:13.444227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372252b-313a-4899-ba45-cc8ae085a5b6 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:13.456546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372252b-313a-4899-ba45-cc8ae085a5b6 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:13.464711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5841572-fb76-4f4d-a01e-38280f0c8c2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:13.465435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5841572-fb76-4f4d-a01e-38280f0c8c2e service swift] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:13.465724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5841572-fb76-4f4d-a01e-38280f0c8c2e service swift] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:13.465999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5841572-fb76-4f4d-a01e-38280f0c8c2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:13.476399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5841572-fb76-4f4d-a01e-38280f0c8c2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:13.478115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5841572-fb76-4f4d-a01e-38280f0c8c2e service swift] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:13.488185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1213/2401] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:00:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:00:23.417931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=9 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:23.483714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=34 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:31.670677 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9d902-f74a-4dbe-9acb-296626501cce None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:31.671117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9d902-f74a-4dbe-9acb-296626501cce None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:31.671215 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9d902-f74a-4dbe-9acb-296626501cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:31.689384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-82c9d902-f74a-4dbe-9acb-296626501cce None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:31.690563 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-82c9d902-f74a-4dbe-9acb-296626501cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:31.697839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1189/2402] 10.222.0.81 () {58 vars in 967 bytes} [Thu Jun 16 19:00:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:31.711242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5841572-fb76-4f4d-a01e-38280f0c8c2e service swift] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:31.721604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5841572-fb76-4f4d-a01e-38280f0c8c2e service swift] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:31.727148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194ca99-1d58-4674-9396-89229e069ab4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:31.727782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194ca99-1d58-4674-9396-89229e069ab4 service swift] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:31.728088 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194ca99-1d58-4674-9396-89229e069ab4 service swift] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:31.728367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194ca99-1d58-4674-9396-89229e069ab4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:31.736425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194ca99-1d58-4674-9396-89229e069ab4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:31.738085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194ca99-1d58-4674-9396-89229e069ab4 service swift] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:31.746572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1214/2403] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:00:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:00:34.767348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9d902-f74a-4dbe-9acb-296626501cce None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:34.768788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-82c9d902-f74a-4dbe-9acb-296626501cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:34.833523 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9d902-f74a-4dbe-9acb-296626501cce None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:34.838672 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c6db8-4a0e-4534-a2fb-5a5f358da5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:34.840740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c6db8-4a0e-4534-a2fb-5a5f358da5c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:34.840740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c6db8-4a0e-4534-a2fb-5a5f358da5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:34.840740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c6db8-4a0e-4534-a2fb-5a5f358da5c1 admin admin] PATH_INFO: `/v3/users/325f963f409b4fd68f9ac9a29de4be72` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:34.841531 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c6db8-4a0e-4534-a2fb-5a5f358da5c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=325f963f409b4fd68f9ac9a29de4be72)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:34.843460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c6db8-4a0e-4534-a2fb-5a5f358da5c1 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:35.009701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-1c4c6db8-4a0e-4534-a2fb-5a5f358da5c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325f963f409b4fd68f9ac9a29de4be72'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:35.016392 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-1c4c6db8-4a0e-4534-a2fb-5a5f358da5c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325f963f409b4fd68f9ac9a29de4be72'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:35.017081 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-1c4c6db8-4a0e-4534-a2fb-5a5f358da5c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325f963f409b4fd68f9ac9a29de4be72'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:35.029439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-1c4c6db8-4a0e-4534-a2fb-5a5f358da5c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325f963f409b4fd68f9ac9a29de4be72'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:35.030100 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-1c4c6db8-4a0e-4534-a2fb-5a5f358da5c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325f963f409b4fd68f9ac9a29de4be72'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:35.098247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1190/2404] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:34 2022] DELETE /identity/v3/users/325f963f409b4fd68f9ac9a29de4be72 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:35.334920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194ca99-1d58-4674-9396-89229e069ab4 service swift] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:35.349413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194ca99-1d58-4674-9396-89229e069ab4 service swift] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:35.408964 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:35.409861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:35.410148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:35.410424 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] PATH_INFO: `/v3/projects/c5e5a9406f7f4ccba4aaf2cccf1de07e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:35.411591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e5a9406f7f4ccba4aaf2cccf1de07e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:35.414035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:35.528602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e5a9406f7f4ccba4aaf2cccf1de07e', 'request_id': 'req-47c5d4e6-7346-401f-a88e-89c750577bc6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:35.601931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e5a9406f7f4ccba4aaf2cccf1de07e', 'request_id': 'req-47c5d4e6-7346-401f-a88e-89c750577bc6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:35.608065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e5a9406f7f4ccba4aaf2cccf1de07e', 'request_id': 'req-47c5d4e6-7346-401f-a88e-89c750577bc6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:35.620043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] The token cache is being invalidate because project c5e5a9406f7f4ccba4aaf2cccf1de07e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:35.620335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:35.620957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-47c5d4e6-7346-401f-a88e-89c750577bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:35.622486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1215/2405] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:35 2022] DELETE /identity/v3/projects/c5e5a9406f7f4ccba4aaf2cccf1de07e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:35.627397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd8871-1cbb-4639-9801-08a1f6d4b3f4 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:35.627718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd8871-1cbb-4639-9801-08a1f6d4b3f4 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:35.628024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd8871-1cbb-4639-9801-08a1f6d4b3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:35.646288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-85dd8871-1cbb-4639-9801-08a1f6d4b3f4 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:35.647515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-85dd8871-1cbb-4639-9801-08a1f6d4b3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:35.706064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1191/2406] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:35.713907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4406d-60b5-4052-aa46-2b0f70e9f5cb None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:35.714198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4406d-60b5-4052-aa46-2b0f70e9f5cb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:35.714502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4406d-60b5-4052-aa46-2b0f70e9f5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:35.732349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3fa4406d-60b5-4052-aa46-2b0f70e9f5cb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:35.733583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3fa4406d-60b5-4052-aa46-2b0f70e9f5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:35.742324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1216/2407] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:35.746344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd8871-1cbb-4639-9801-08a1f6d4b3f4 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:35.756065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd8871-1cbb-4639-9801-08a1f6d4b3f4 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:35.761180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29b487-dcc8-44eb-8a12-8c8cbda0ecac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:35.761886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29b487-dcc8-44eb-8a12-8c8cbda0ecac admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:35.762180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29b487-dcc8-44eb-8a12-8c8cbda0ecac admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:35.762457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29b487-dcc8-44eb-8a12-8c8cbda0ecac admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:35.763171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29b487-dcc8-44eb-8a12-8c8cbda0ecac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:00:35.763481 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29b487-dcc8-44eb-8a12-8c8cbda0ecac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:35.765085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29b487-dcc8-44eb-8a12-8c8cbda0ecac admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:35.773480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:35.776638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1192/2408] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:00:35.780848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4406d-60b5-4052-aa46-2b0f70e9f5cb None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:35.790078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4406d-60b5-4052-aa46-2b0f70e9f5cb None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:35.794654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b17180-3683-4505-85ab-d2d2d9de3bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:35.795291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b17180-3683-4505-85ab-d2d2d9de3bda admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:35.795575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b17180-3683-4505-85ab-d2d2d9de3bda admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:35.795849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b17180-3683-4505-85ab-d2d2d9de3bda admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:35.796524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b17180-3683-4505-85ab-d2d2d9de3bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:35.798470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b17180-3683-4505-85ab-d2d2d9de3bda admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:35.946781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1217/2409] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:00:35 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:35.952212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29b487-dcc8-44eb-8a12-8c8cbda0ecac admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:35.962174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29b487-dcc8-44eb-8a12-8c8cbda0ecac admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:36.018406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38519e-f146-4494-8bfa-c9a78c992aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:36.019079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38519e-f146-4494-8bfa-c9a78c992aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:36.019362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38519e-f146-4494-8bfa-c9a78c992aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:36.019643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38519e-f146-4494-8bfa-c9a78c992aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:36.020322 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38519e-f146-4494-8bfa-c9a78c992aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:36.022405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38519e-f146-4494-8bfa-c9a78c992aa5 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:36.110017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1193/2410] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:00:35 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:00:36.114417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b17180-3683-4505-85ab-d2d2d9de3bda admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:36.125224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b17180-3683-4505-85ab-d2d2d9de3bda admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:36.130140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a10ac-4402-4219-9e69-97cc1fa4b239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:36.130831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a10ac-4402-4219-9e69-97cc1fa4b239 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:36.131118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a10ac-4402-4219-9e69-97cc1fa4b239 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:36.131397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a10ac-4402-4219-9e69-97cc1fa4b239 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:36.132159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a10ac-4402-4219-9e69-97cc1fa4b239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:36.133807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a10ac-4402-4219-9e69-97cc1fa4b239 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:36.144142 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1218/2411] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:36.147922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38519e-f146-4494-8bfa-c9a78c992aa5 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:36.158019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38519e-f146-4494-8bfa-c9a78c992aa5 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:36.162588 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d44ce-d7fc-4f3b-bff5-62937a25d59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:36.163741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d44ce-d7fc-4f3b-bff5-62937a25d59e admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:36.164029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d44ce-d7fc-4f3b-bff5-62937a25d59e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:36.164306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d44ce-d7fc-4f3b-bff5-62937a25d59e admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:36.165044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d44ce-d7fc-4f3b-bff5-62937a25d59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:36.166684 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d44ce-d7fc-4f3b-bff5-62937a25d59e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:36.176365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1194/2412] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:00:36.180865 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a10ac-4402-4219-9e69-97cc1fa4b239 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:36.190700 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a10ac-4402-4219-9e69-97cc1fa4b239 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:36.195273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b101a0d-791c-4600-8b67-16e36cdc407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:36.196146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b101a0d-791c-4600-8b67-16e36cdc407d admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:36.196431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b101a0d-791c-4600-8b67-16e36cdc407d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:36.196711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b101a0d-791c-4600-8b67-16e36cdc407d admin admin] PATH_INFO: `/v3/projects/c11dc9c4eab24553b225ce919fc75d03/users/3a0754efe4b74105a94a77a06687e579/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:36.216835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b101a0d-791c-4600-8b67-16e36cdc407d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c11dc9c4eab24553b225ce919fc75d03, user_id=3a0754efe4b74105a94a77a06687e579, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:36.219498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b101a0d-791c-4600-8b67-16e36cdc407d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:36.282184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1219/2413] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:00:36 2022] PUT /identity/v3/projects/c11dc9c4eab24553b225ce919fc75d03/users/3a0754efe4b74105a94a77a06687e579/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:36.297078 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43595ac-4fb3-455e-84d6-4957d3cb2afc None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:36.297520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43595ac-4fb3-455e-84d6-4957d3cb2afc None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:36.297806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43595ac-4fb3-455e-84d6-4957d3cb2afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:36.324487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-a43595ac-4fb3-455e-84d6-4957d3cb2afc None None] MFA Rules not processed for user `3a0754efe4b74105a94a77a06687e579`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:36.325745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a43595ac-4fb3-455e-84d6-4957d3cb2afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:36.389183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1195/2414] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:00:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:00:36.400854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b101a0d-791c-4600-8b67-16e36cdc407d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:36.402539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0b101a0d-791c-4600-8b67-16e36cdc407d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:36.420270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:36.479400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b101a0d-791c-4600-8b67-16e36cdc407d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:36.483597 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f598ec3-0399-4508-8f72-046a100c9293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:36.484230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598ec3-0399-4508-8f72-046a100c9293 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:36.484512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598ec3-0399-4508-8f72-046a100c9293 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:36.484788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f598ec3-0399-4508-8f72-046a100c9293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:36.493166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f598ec3-0399-4508-8f72-046a100c9293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:36.494611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f598ec3-0399-4508-8f72-046a100c9293 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:36.503168 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1220/2415] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:43.642378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43595ac-4fb3-455e-84d6-4957d3cb2afc None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:43.711949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43595ac-4fb3-455e-84d6-4957d3cb2afc None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:43.716492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:43.717399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:43.717702 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:43.717968 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] PATH_INFO: `/v3/users/3a0754efe4b74105a94a77a06687e579` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:43.719179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a0754efe4b74105a94a77a06687e579)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:43.721159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:43.733971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:43.796200 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e2311-8f4a-4a44-ac84-29ba849e5eb2 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:43.796705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e2311-8f4a-4a44-ac84-29ba849e5eb2 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:43.796825 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e2311-8f4a-4a44-ac84-29ba849e5eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:43.814035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-312e2311-8f4a-4a44-ac84-29ba849e5eb2 None None] MFA Rules not processed for user `7310c631282c495a83e17dbdbc6d2ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:43.815226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-312e2311-8f4a-4a44-ac84-29ba849e5eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:43.871235 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1221/2416] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:00:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:43.958858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a0754efe4b74105a94a77a06687e579'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:43.966679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a0754efe4b74105a94a77a06687e579'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:43.967374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a0754efe4b74105a94a77a06687e579'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:43.980829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a0754efe4b74105a94a77a06687e579'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:43.981564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a0754efe4b74105a94a77a06687e579'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:44.071472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1196/2417] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:00:43 2022] DELETE /identity/v3/users/3a0754efe4b74105a94a77a06687e579 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:44.076073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aadf82-9d5c-4442-aa00-2bc50ace8296 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:44.080987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aadf82-9d5c-4442-aa00-2bc50ace8296 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:44.080987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aadf82-9d5c-4442-aa00-2bc50ace8296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:44.094382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f0aadf82-9d5c-4442-aa00-2bc50ace8296 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:00:44.095733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f0aadf82-9d5c-4442-aa00-2bc50ace8296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:44.153881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1222/2418] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:00:44.163818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:44.165256 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:44.221858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3039dc-7e7c-4c1c-91b8-a5a33e6fe49e admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:44.225494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e502a-59d4-404a-ae63-fe969e41d798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:44.226133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e502a-59d4-404a-ae63-fe969e41d798 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:44.226661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e502a-59d4-404a-ae63-fe969e41d798 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:44.226783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e502a-59d4-404a-ae63-fe969e41d798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:44.235985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e502a-59d4-404a-ae63-fe969e41d798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:44.241663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e502a-59d4-404a-ae63-fe969e41d798 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:44.247016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1197/2419] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:00:44.770132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aadf82-9d5c-4442-aa00-2bc50ace8296 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:00:44.773445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f0aadf82-9d5c-4442-aa00-2bc50ace8296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:00:44.798030 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aadf82-9d5c-4442-aa00-2bc50ace8296 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:00:44.803769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:00:44.805225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:00:44.805538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:00:44.805812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] PATH_INFO: `/v3/projects/c11dc9c4eab24553b225ce919fc75d03` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:00:44.807149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c11dc9c4eab24553b225ce919fc75d03)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:00:44.809172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:00:44.943989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:44.952463 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c11dc9c4eab24553b225ce919fc75d03', 'request_id': 'req-db135cd0-7b3b-4175-8f44-2113b02e8e44'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.020318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c11dc9c4eab24553b225ce919fc75d03', 'request_id': 'req-db135cd0-7b3b-4175-8f44-2113b02e8e44'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.026356 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c11dc9c4eab24553b225ce919fc75d03', 'request_id': 'req-db135cd0-7b3b-4175-8f44-2113b02e8e44'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.037462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] The token cache is being invalidate because project c11dc9c4eab24553b225ce919fc75d03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:00:45.037753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.038396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:00:45.040492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1223/2420] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:00:44 2022] DELETE /identity/v3/projects/c11dc9c4eab24553b225ce919fc75d03 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:00:54.242848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:DELETE=10,keystone:SELECT=48,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:00:55.140019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=11,keystone:DELETE=4,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:07.673525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e502a-59d4-404a-ae63-fe969e41d798 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:07.674966 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-4e3e502a-59d4-404a-ae63-fe969e41d798 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:07.734713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e502a-59d4-404a-ae63-fe969e41d798 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:07.783657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140220f-7e11-4acf-8978-829c6dd54f3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:07.784333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140220f-7e11-4acf-8978-829c6dd54f3d service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:07.784618 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140220f-7e11-4acf-8978-829c6dd54f3d service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:07.784893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140220f-7e11-4acf-8978-829c6dd54f3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:07.786746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-3140220f-7e11-4acf-8978-829c6dd54f3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:07.845044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140220f-7e11-4acf-8978-829c6dd54f3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:07.846510 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140220f-7e11-4acf-8978-829c6dd54f3d service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:07.856215 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1198/2421] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:08.328874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:08.330327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:08.402484 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db135cd0-7b3b-4175-8f44-2113b02e8e44 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:08.406234 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580cb0ba-64f3-4312-9e94-af62ab9e2123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:08.406884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cb0ba-64f3-4312-9e94-af62ab9e2123 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:08.407171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cb0ba-64f3-4312-9e94-af62ab9e2123 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:08.407453 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cb0ba-64f3-4312-9e94-af62ab9e2123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:08.409308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-580cb0ba-64f3-4312-9e94-af62ab9e2123 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:08.480613 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cb0ba-64f3-4312-9e94-af62ab9e2123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:08.482121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580cb0ba-64f3-4312-9e94-af62ab9e2123 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:08.500837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1224/2422] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:11.191413 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140220f-7e11-4acf-8978-829c6dd54f3d service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:11.198648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140220f-7e11-4acf-8978-829c6dd54f3d service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:11.202424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f8372-b3cc-4967-a508-e2cb992b86b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:11.203151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f8372-b3cc-4967-a508-e2cb992b86b5 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:11.203456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f8372-b3cc-4967-a508-e2cb992b86b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:11.203736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f8372-b3cc-4967-a508-e2cb992b86b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:11.205584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-395f8372-b3cc-4967-a508-e2cb992b86b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:11.219190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f8372-b3cc-4967-a508-e2cb992b86b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:11.220611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f8372-b3cc-4967-a508-e2cb992b86b5 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:11.232298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1199/2423] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:12.383067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60e5fb-a41b-4b81-b939-3df5d2f1e1ce None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:12.383609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60e5fb-a41b-4b81-b939-3df5d2f1e1ce None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:12.383761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f60e5fb-a41b-4b81-b939-3df5d2f1e1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:12.401505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0f60e5fb-a41b-4b81-b939-3df5d2f1e1ce None None] MFA Rules not processed for user `2de523abbc7b4f8b972dc97b8a013237`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:12.403334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0f60e5fb-a41b-4b81-b939-3df5d2f1e1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:12.459352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1225/2424] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:01:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:12.471370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f8372-b3cc-4967-a508-e2cb992b86b5 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:12.472919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-395f8372-b3cc-4967-a508-e2cb992b86b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:12.487032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f8372-b3cc-4967-a508-e2cb992b86b5 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:12.492907 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57f3e7-867d-4611-a501-d9754e204203 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:12.493684 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57f3e7-867d-4611-a501-d9754e204203 service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:12.494041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57f3e7-867d-4611-a501-d9754e204203 service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:12.494388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57f3e7-867d-4611-a501-d9754e204203 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:12.496208 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1a57f3e7-867d-4611-a501-d9754e204203 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:12.507221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57f3e7-867d-4611-a501-d9754e204203 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:12.508664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57f3e7-867d-4611-a501-d9754e204203 service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:12.520510 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1200/2425] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:12.948092 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60e5fb-a41b-4b81-b939-3df5d2f1e1ce None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:12.958689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f60e5fb-a41b-4b81-b939-3df5d2f1e1ce None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:12.962685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32cdc9-2a69-449c-862b-c5c0eb01a0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:12.966073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32cdc9-2a69-449c-862b-c5c0eb01a0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:12.966468 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32cdc9-2a69-449c-862b-c5c0eb01a0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:12.966849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32cdc9-2a69-449c-862b-c5c0eb01a0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:12.969066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0b32cdc9-2a69-449c-862b-c5c0eb01a0c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:12.980165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32cdc9-2a69-449c-862b-c5c0eb01a0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:12.981738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32cdc9-2a69-449c-862b-c5c0eb01a0c8 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:12.990940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1226/2426] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:13.586350 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57f3e7-867d-4611-a501-d9754e204203 service nova] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:13.587790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1a57f3e7-867d-4611-a501-d9754e204203 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:13.637177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:13.670196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57f3e7-867d-4611-a501-d9754e204203 service nova] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:13.675476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19726d2-1aca-4d95-9fbc-1542d64c18ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:13.680172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19726d2-1aca-4d95-9fbc-1542d64c18ac service placement] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:13.680172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19726d2-1aca-4d95-9fbc-1542d64c18ac service placement] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:13.680172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19726d2-1aca-4d95-9fbc-1542d64c18ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:13.680172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b19726d2-1aca-4d95-9fbc-1542d64c18ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:13.692176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19726d2-1aca-4d95-9fbc-1542d64c18ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:13.693826 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19726d2-1aca-4d95-9fbc-1542d64c18ac service placement] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:13.704850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1201/2427] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:16.476499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c6889-a364-4266-a4bb-569a203a3323 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:16.483599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c6889-a364-4266-a4bb-569a203a3323 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:16.483599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c6889-a364-4266-a4bb-569a203a3323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:16.499496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-796c6889-a364-4266-a4bb-569a203a3323 None None] MFA Rules not processed for user `fec4e3ac2e44403b9d45b37b5a6f64a9`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:16.500192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-796c6889-a364-4266-a4bb-569a203a3323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:16.507975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1227/2428] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:01:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:19.845454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19726d2-1aca-4d95-9fbc-1542d64c18ac service placement] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:19.848668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b19726d2-1aca-4d95-9fbc-1542d64c18ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:19.925688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19726d2-1aca-4d95-9fbc-1542d64c18ac service placement] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:19.930304 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:19.934340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:19.934340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:19.934340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] PATH_INFO: `/v3/users/eed09fa6ecf14635949417f19c8a3af0` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:19.934340 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eed09fa6ecf14635949417f19c8a3af0)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:19.934823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:20.188962 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eed09fa6ecf14635949417f19c8a3af0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:20.195625 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eed09fa6ecf14635949417f19c8a3af0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:20.196272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eed09fa6ecf14635949417f19c8a3af0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:20.208277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eed09fa6ecf14635949417f19c8a3af0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:20.208984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eed09fa6ecf14635949417f19c8a3af0'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:20.272848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1202/2429] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:19 2022] DELETE /identity/v3/users/eed09fa6ecf14635949417f19c8a3af0 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:20.499095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c6889-a364-4266-a4bb-569a203a3323 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:20.510406 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c6889-a364-4266-a4bb-569a203a3323 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:20.539165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:20.572710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:20.573615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:20.573897 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:20.574170 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] PATH_INFO: `/v3/projects/e16a74b1bc56455b8f60bfc28bc94afb` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:20.576003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e16a74b1bc56455b8f60bfc28bc94afb)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:20.577953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:20.695067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e16a74b1bc56455b8f60bfc28bc94afb', 'request_id': 'req-2d4693d6-b151-470e-943f-f53f689a4227'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:20.748179 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e16a74b1bc56455b8f60bfc28bc94afb', 'request_id': 'req-2d4693d6-b151-470e-943f-f53f689a4227'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:20.757679 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e16a74b1bc56455b8f60bfc28bc94afb', 'request_id': 'req-2d4693d6-b151-470e-943f-f53f689a4227'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:20.774076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] The token cache is being invalidate because project e16a74b1bc56455b8f60bfc28bc94afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:20.778091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:20.778091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-2d4693d6-b151-470e-943f-f53f689a4227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:20.778091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1228/2430] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:20 2022] DELETE /identity/v3/projects/e16a74b1bc56455b8f60bfc28bc94afb => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:21.440735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:21.442277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:21.502326 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67bf04-b754-4002-9e8e-e6ca872d0dc7 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:21.509735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:21.555846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6e16c-8603-4434-a300-bbb73ce6a748 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:21.555846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6e16c-8603-4434-a300-bbb73ce6a748 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:21.555846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6e16c-8603-4434-a300-bbb73ce6a748 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:21.555846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6e16c-8603-4434-a300-bbb73ce6a748 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:21.556310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e0f6e16c-8603-4434-a300-bbb73ce6a748 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:21.564163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6e16c-8603-4434-a300-bbb73ce6a748 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:21.565636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6e16c-8603-4434-a300-bbb73ce6a748 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:21.574040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1203/2431] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:26.653377 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaea525-68e2-4f1c-9ea9-0f9680cd3fbc None None] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.661685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaea525-68e2-4f1c-9ea9-0f9680cd3fbc None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.661685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaea525-68e2-4f1c-9ea9-0f9680cd3fbc None None] PATH_INFO: `/` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.661685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1229/2432] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 19:01:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:01:26.662040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e4792-5163-4bb9-a368-51d5f466dbdb None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.662040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e4792-5163-4bb9-a368-51d5f466dbdb None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.662040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e4792-5163-4bb9-a368-51d5f466dbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.680641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-104e4792-5163-4bb9-a368-51d5f466dbdb None None] MFA Rules not processed for user `f4e05d75cd1142e18d7518b78edd88fa`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:26.681895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-104e4792-5163-4bb9-a368-51d5f466dbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:26.690869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1204/2433] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 19:01:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:01:26.702633 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaea525-68e2-4f1c-9ea9-0f9680cd3fbc None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:26.704557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-1eaea525-68e2-4f1c-9ea9-0f9680cd3fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:26.727531 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaea525-68e2-4f1c-9ea9-0f9680cd3fbc None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:26.731155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1014b4b-a096-4ba4-9e7a-a36511597abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:26.732327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1014b4b-a096-4ba4-9e7a-a36511597abe service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.732615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1014b4b-a096-4ba4-9e7a-a36511597abe service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.732895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1014b4b-a096-4ba4-9e7a-a36511597abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.740092 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1014b4b-a096-4ba4-9e7a-a36511597abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:26.741635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1014b4b-a096-4ba4-9e7a-a36511597abe service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.750276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1230/2434] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:26.838927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee718b-f409-4905-bd7a-e996cad36208 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.839288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee718b-f409-4905-bd7a-e996cad36208 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.841885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee718b-f409-4905-bd7a-e996cad36208 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.841885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1205/2435] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 19:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:01:26.847147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24985a-82bc-4c18-b363-6abf8624accc None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.847579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24985a-82bc-4c18-b363-6abf8624accc None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.847858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24985a-82bc-4c18-b363-6abf8624accc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.866398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ed24985a-82bc-4c18-b363-6abf8624accc None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:26.867590 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ed24985a-82bc-4c18-b363-6abf8624accc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:26.934863 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1231/2436] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 19:01:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:26.945352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee718b-f409-4905-bd7a-e996cad36208 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:26.957087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee718b-f409-4905-bd7a-e996cad36208 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:26.960664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e144b-071e-4680-9302-5840dfb0fff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:26.961355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e144b-071e-4680-9302-5840dfb0fff6 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:26.961643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e144b-071e-4680-9302-5840dfb0fff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:26.972301 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e144b-071e-4680-9302-5840dfb0fff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:26.975125 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e144b-071e-4680-9302-5840dfb0fff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:26.976566 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e144b-071e-4680-9302-5840dfb0fff6 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:26.989035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1206/2437] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:28.707158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24985a-82bc-4c18-b363-6abf8624accc None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:28.707158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ed24985a-82bc-4c18-b363-6abf8624accc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:28.722468 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24985a-82bc-4c18-b363-6abf8624accc None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:28.728187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20af87-8387-46be-aa81-34098d78975b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:28.729035 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20af87-8387-46be-aa81-34098d78975b service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:28.729286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20af87-8387-46be-aa81-34098d78975b service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:28.729405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20af87-8387-46be-aa81-34098d78975b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:28.731300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-eb20af87-8387-46be-aa81-34098d78975b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:28.744317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20af87-8387-46be-aa81-34098d78975b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:28.745992 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20af87-8387-46be-aa81-34098d78975b service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:28.758372 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1232/2438] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:29.597825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e144b-071e-4680-9302-5840dfb0fff6 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:29.599277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a62e144b-071e-4680-9302-5840dfb0fff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:29.672699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e144b-071e-4680-9302-5840dfb0fff6 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:29.676460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079948d9-cb9a-408f-9386-bdd55e8f3974 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:29.677368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079948d9-cb9a-408f-9386-bdd55e8f3974 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:29.677803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079948d9-cb9a-408f-9386-bdd55e8f3974 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:29.678242 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079948d9-cb9a-408f-9386-bdd55e8f3974 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:29.680369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-079948d9-cb9a-408f-9386-bdd55e8f3974 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:29.798588 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079948d9-cb9a-408f-9386-bdd55e8f3974 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:29.800402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079948d9-cb9a-408f-9386-bdd55e8f3974 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:29.821434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1207/2439] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:01:38.753369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:39.808703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=79 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:53.648000 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20af87-8387-46be-aa81-34098d78975b service nova] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:53.657077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20af87-8387-46be-aa81-34098d78975b service nova] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:53.661152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ca7f6-9167-42fa-8345-db9da11094b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:53.661816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ca7f6-9167-42fa-8345-db9da11094b7 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:53.662097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ca7f6-9167-42fa-8345-db9da11094b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:53.662372 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ca7f6-9167-42fa-8345-db9da11094b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:53.664103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-641ca7f6-9167-42fa-8345-db9da11094b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:53.735492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ca7f6-9167-42fa-8345-db9da11094b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:53.736971 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ca7f6-9167-42fa-8345-db9da11094b7 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:53.747635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1233/2440] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:01:54.672254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f3e8e-6ce4-4c24-aadb-dcda779473ca None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:54.672254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f3e8e-6ce4-4c24-aadb-dcda779473ca None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:54.672254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f3e8e-6ce4-4c24-aadb-dcda779473ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:54.686880 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-7b4f3e8e-6ce4-4c24-aadb-dcda779473ca None None] MFA Rules not processed for user `1c0e7e9f9d0f409a8820d8bd8dd239cf`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:54.688075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-7b4f3e8e-6ce4-4c24-aadb-dcda779473ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:54.754629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1208/2441] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:01:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:57.766244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ca7f6-9167-42fa-8345-db9da11094b7 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:57.767853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-641ca7f6-9167-42fa-8345-db9da11094b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:57.780479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ca7f6-9167-42fa-8345-db9da11094b7 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:57.785365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49794f8e-1441-4f20-9602-c51fdb70a635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:57.786234 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49794f8e-1441-4f20-9602-c51fdb70a635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:57.790649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49794f8e-1441-4f20-9602-c51fdb70a635 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:57.790649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49794f8e-1441-4f20-9602-c51fdb70a635 admin admin] PATH_INFO: `/v3/users/7310c631282c495a83e17dbdbc6d2ba0` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:57.790649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49794f8e-1441-4f20-9602-c51fdb70a635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7310c631282c495a83e17dbdbc6d2ba0)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:57.790649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49794f8e-1441-4f20-9602-c51fdb70a635 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:57.954062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-49794f8e-1441-4f20-9602-c51fdb70a635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7310c631282c495a83e17dbdbc6d2ba0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:57.955408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-49794f8e-1441-4f20-9602-c51fdb70a635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7310c631282c495a83e17dbdbc6d2ba0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:57.962577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-49794f8e-1441-4f20-9602-c51fdb70a635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7310c631282c495a83e17dbdbc6d2ba0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:57.963230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-49794f8e-1441-4f20-9602-c51fdb70a635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7310c631282c495a83e17dbdbc6d2ba0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:57.976537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-49794f8e-1441-4f20-9602-c51fdb70a635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7310c631282c495a83e17dbdbc6d2ba0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:58.034828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1234/2442] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:01:57 2022] DELETE /identity/v3/users/7310c631282c495a83e17dbdbc6d2ba0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:58.275017 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f3e8e-6ce4-4c24-aadb-dcda779473ca None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:58.286443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f3e8e-6ce4-4c24-aadb-dcda779473ca None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:58.358584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:58.367846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:58.368209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:58.368497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] PATH_INFO: `/v3/projects/ed8501d1ff734054ba0a6ef4c398d051` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:58.369870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed8501d1ff734054ba0a6ef4c398d051)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:58.372118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:58.473484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed8501d1ff734054ba0a6ef4c398d051', 'request_id': 'req-797542c3-e9ad-411c-a56b-b6e42df3b60e'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:58.527012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed8501d1ff734054ba0a6ef4c398d051', 'request_id': 'req-797542c3-e9ad-411c-a56b-b6e42df3b60e'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:58.533532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed8501d1ff734054ba0a6ef4c398d051', 'request_id': 'req-797542c3-e9ad-411c-a56b-b6e42df3b60e'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:58.556581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] The token cache is being invalidate because project ed8501d1ff734054ba0a6ef4c398d051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:01:58.557650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:58.557948 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-797542c3-e9ad-411c-a56b-b6e42df3b60e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:01:58.559788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1209/2443] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:01:58 2022] DELETE /identity/v3/projects/ed8501d1ff734054ba0a6ef4c398d051 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:58.564990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6bf74-cd92-4b06-97c4-a1d902e46fce None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:58.566718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6bf74-cd92-4b06-97c4-a1d902e46fce None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:58.570851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6bf74-cd92-4b06-97c4-a1d902e46fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:58.586379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0ad6bf74-cd92-4b06-97c4-a1d902e46fce None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:58.587828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0ad6bf74-cd92-4b06-97c4-a1d902e46fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:58.653153 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1235/2444] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:58.665549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb3c41-9070-456a-a8d2-d62556a28096 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:58.665995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb3c41-9070-456a-a8d2-d62556a28096 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:58.666426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb3c41-9070-456a-a8d2-d62556a28096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:58.685598 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-eecb3c41-9070-456a-a8d2-d62556a28096 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:58.686792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-eecb3c41-9070-456a-a8d2-d62556a28096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:58.695322 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1210/2445] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:58.699975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6bf74-cd92-4b06-97c4-a1d902e46fce None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:58.709950 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6bf74-cd92-4b06-97c4-a1d902e46fce None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:58.714698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c23df-aecf-4094-b234-6be6dc72a3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:58.715353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c23df-aecf-4094-b234-6be6dc72a3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:58.715640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c23df-aecf-4094-b234-6be6dc72a3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:58.715916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c23df-aecf-4094-b234-6be6dc72a3f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:58.716634 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c23df-aecf-4094-b234-6be6dc72a3f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:01:58.716944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c23df-aecf-4094-b234-6be6dc72a3f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:58.718581 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c23df-aecf-4094-b234-6be6dc72a3f7 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:58.720257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:58.732331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1236/2446] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:01:58.736616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb3c41-9070-456a-a8d2-d62556a28096 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:58.746916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb3c41-9070-456a-a8d2-d62556a28096 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:58.751769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b04b8-e69e-480f-b000-5a852490ecb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:58.752509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b04b8-e69e-480f-b000-5a852490ecb7 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:58.752890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b04b8-e69e-480f-b000-5a852490ecb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:58.753198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b04b8-e69e-480f-b000-5a852490ecb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:58.753920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b04b8-e69e-480f-b000-5a852490ecb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:58.755834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b04b8-e69e-480f-b000-5a852490ecb7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:58.849641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1211/2447] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:58 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:01:58.853874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c23df-aecf-4094-b234-6be6dc72a3f7 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:58.864250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c23df-aecf-4094-b234-6be6dc72a3f7 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:58.921469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac8873-6a38-424e-9d5a-e5f25a0c04a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:58.922197 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac8873-6a38-424e-9d5a-e5f25a0c04a1 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:58.922518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac8873-6a38-424e-9d5a-e5f25a0c04a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:58.922792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac8873-6a38-424e-9d5a-e5f25a0c04a1 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:58.923467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac8873-6a38-424e-9d5a-e5f25a0c04a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:58.925655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac8873-6a38-424e-9d5a-e5f25a0c04a1 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:58.996221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1237/2448] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:58 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:01:59.000418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b04b8-e69e-480f-b000-5a852490ecb7 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:59.010630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b04b8-e69e-480f-b000-5a852490ecb7 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:59.015628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89234034-53c6-4fdf-ae27-5f753e106a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:59.016423 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89234034-53c6-4fdf-ae27-5f753e106a2d admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:59.016815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89234034-53c6-4fdf-ae27-5f753e106a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:59.017031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89234034-53c6-4fdf-ae27-5f753e106a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:59.018021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89234034-53c6-4fdf-ae27-5f753e106a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:59.020091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89234034-53c6-4fdf-ae27-5f753e106a2d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:59.021738 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:59.032466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1212/2449] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:59.037223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac8873-6a38-424e-9d5a-e5f25a0c04a1 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:59.054284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac8873-6a38-424e-9d5a-e5f25a0c04a1 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:59.054284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825c743-b229-4bbe-b6f3-f282209f2fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:59.054284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825c743-b229-4bbe-b6f3-f282209f2fed admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:59.054284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825c743-b229-4bbe-b6f3-f282209f2fed admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:59.054863 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825c743-b229-4bbe-b6f3-f282209f2fed admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:59.055075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825c743-b229-4bbe-b6f3-f282209f2fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:59.056582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825c743-b229-4bbe-b6f3-f282209f2fed admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:59.076075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1238/2450] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:59.077247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89234034-53c6-4fdf-ae27-5f753e106a2d admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:59.090201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89234034-53c6-4fdf-ae27-5f753e106a2d admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:59.098066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a4ba3-3f07-4f31-9677-7a337d66d863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:59.108117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4ba3-3f07-4f31-9677-7a337d66d863 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:59.108117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4ba3-3f07-4f31-9677-7a337d66d863 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:59.108117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4ba3-3f07-4f31-9677-7a337d66d863 admin admin] PATH_INFO: `/v3/projects/6e8fe952ec9c44c1a62e4921d343c5f4/users/0f28795e34204832a8ac634e0ebd5aae/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:59.117966 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a4ba3-3f07-4f31-9677-7a337d66d863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8fe952ec9c44c1a62e4921d343c5f4, user_id=0f28795e34204832a8ac634e0ebd5aae, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:59.121145 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a4ba3-3f07-4f31-9677-7a337d66d863 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:59.260300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1213/2451] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:59 2022] PUT /identity/v3/projects/6e8fe952ec9c44c1a62e4921d343c5f4/users/0f28795e34204832a8ac634e0ebd5aae/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:59.271183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedf5e3-e71d-4cec-bda1-d4cd03139792 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:59.271557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedf5e3-e71d-4cec-bda1-d4cd03139792 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:59.271921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedf5e3-e71d-4cec-bda1-d4cd03139792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:59.303825 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0dedf5e3-e71d-4cec-bda1-d4cd03139792 None None] MFA Rules not processed for user `0f28795e34204832a8ac634e0ebd5aae`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:01:59.305178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0dedf5e3-e71d-4cec-bda1-d4cd03139792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:01:59.403008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1239/2452] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:01:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:01:59.407715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a4ba3-3f07-4f31-9677-7a337d66d863 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:59.475674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a4ba3-3f07-4f31-9677-7a337d66d863 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:59.480305 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a24030-24cb-422a-a4f9-f74f50365dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:59.480968 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24030-24cb-422a-a4f9-f74f50365dbd admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:59.481308 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24030-24cb-422a-a4f9-f74f50365dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:59.481586 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24030-24cb-422a-a4f9-f74f50365dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:59.482260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a24030-24cb-422a-a4f9-f74f50365dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:59.484159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a24030-24cb-422a-a4f9-f74f50365dbd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:59.601005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1214/2453] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:01:59 2022] POST /identity/v3/projects => generated 377 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:59.602244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedf5e3-e71d-4cec-bda1-d4cd03139792 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:59.614022 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedf5e3-e71d-4cec-bda1-d4cd03139792 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:59.674605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056e774-0192-4284-b0b3-fc2013010c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:59.675647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056e774-0192-4284-b0b3-fc2013010c70 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:59.675647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056e774-0192-4284-b0b3-fc2013010c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:59.675910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056e774-0192-4284-b0b3-fc2013010c70 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:59.676730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056e774-0192-4284-b0b3-fc2013010c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:59.678901 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056e774-0192-4284-b0b3-fc2013010c70 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:59.687361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:01:59.754552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1240/2454] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:01:59 2022] POST /identity/v3/users => generated 295 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:01:59.755912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a24030-24cb-422a-a4f9-f74f50365dbd admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:59.766295 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a24030-24cb-422a-a4f9-f74f50365dbd admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:59.771591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c7eee-a01c-4c84-a59a-5e7cc9035681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:59.772322 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c7eee-a01c-4c84-a59a-5e7cc9035681 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:59.772548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c7eee-a01c-4c84-a59a-5e7cc9035681 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:59.772828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c7eee-a01c-4c84-a59a-5e7cc9035681 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:59.773590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c7eee-a01c-4c84-a59a-5e7cc9035681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:59.776638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c7eee-a01c-4c84-a59a-5e7cc9035681 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:59.787521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1215/2455] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:01:59.791716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056e774-0192-4284-b0b3-fc2013010c70 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:59.805774 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056e774-0192-4284-b0b3-fc2013010c70 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:01:59.812073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f337a-2671-4d78-839d-facaffc43fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:01:59.812896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f337a-2671-4d78-839d-facaffc43fe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:01:59.813186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f337a-2671-4d78-839d-facaffc43fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:01:59.813491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f337a-2671-4d78-839d-facaffc43fe6 admin admin] PATH_INFO: `/v3/projects/8d7cdfb8d29a446293c66ef7fbb3ed66/users/70abad4ef7e84cacb05f1e14ee70f4c5/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:01:59.834739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f337a-2671-4d78-839d-facaffc43fe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7cdfb8d29a446293c66ef7fbb3ed66, user_id=70abad4ef7e84cacb05f1e14ee70f4c5, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:01:59.837711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f337a-2671-4d78-839d-facaffc43fe6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:01:59.933052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1241/2456] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:01:59 2022] PUT /identity/v3/projects/8d7cdfb8d29a446293c66ef7fbb3ed66/users/70abad4ef7e84cacb05f1e14ee70f4c5/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:01:59.937383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c7eee-a01c-4c84-a59a-5e7cc9035681 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:01:59.947399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c7eee-a01c-4c84-a59a-5e7cc9035681 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:00.006254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b815499-78f3-4cbd-a4cc-c38f26e3a02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:00.006933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b815499-78f3-4cbd-a4cc-c38f26e3a02e admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:00.007223 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b815499-78f3-4cbd-a4cc-c38f26e3a02e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:00.007660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b815499-78f3-4cbd-a4cc-c38f26e3a02e admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:00.008390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b815499-78f3-4cbd-a4cc-c38f26e3a02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:00.010109 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b815499-78f3-4cbd-a4cc-c38f26e3a02e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:00.021845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1216/2457] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:00.030308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f337a-2671-4d78-839d-facaffc43fe6 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:00.042119 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f337a-2671-4d78-839d-facaffc43fe6 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:00.047177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac9d1c-654e-4d85-b9a8-d65c4ce7d3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:00.048131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac9d1c-654e-4d85-b9a8-d65c4ce7d3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:00.048486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac9d1c-654e-4d85-b9a8-d65c4ce7d3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:00.048876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac9d1c-654e-4d85-b9a8-d65c4ce7d3d6 admin admin] PATH_INFO: `/v3/projects/8d7cdfb8d29a446293c66ef7fbb3ed66/users/70abad4ef7e84cacb05f1e14ee70f4c5/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:00.052002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac9d1c-654e-4d85-b9a8-d65c4ce7d3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7cdfb8d29a446293c66ef7fbb3ed66, user_id=70abad4ef7e84cacb05f1e14ee70f4c5, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:00.054744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac9d1c-654e-4d85-b9a8-d65c4ce7d3d6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:00.069291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1242/2458] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:00 2022] PUT /identity/v3/projects/8d7cdfb8d29a446293c66ef7fbb3ed66/users/70abad4ef7e84cacb05f1e14ee70f4c5/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:00.081986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e44e90-8b59-4354-8ddc-cbd2ed262f06 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:00.082263 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e44e90-8b59-4354-8ddc-cbd2ed262f06 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:00.082549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e44e90-8b59-4354-8ddc-cbd2ed262f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:00.112640 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b9e44e90-8b59-4354-8ddc-cbd2ed262f06 None None] MFA Rules not processed for user `70abad4ef7e84cacb05f1e14ee70f4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:00.113991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b9e44e90-8b59-4354-8ddc-cbd2ed262f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:00.148329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:00.213993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1217/2459] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:00.222002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac9d1c-654e-4d85-b9a8-d65c4ce7d3d6 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:00.224158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e8ac9d1c-654e-4d85-b9a8-d65c4ce7d3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:00.288104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac9d1c-654e-4d85-b9a8-d65c4ce7d3d6 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:00.291811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393335e-2df6-4c73-8e3e-ee7e0ee70cff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:00.292673 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393335e-2df6-4c73-8e3e-ee7e0ee70cff service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:00.292797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393335e-2df6-4c73-8e3e-ee7e0ee70cff service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:00.293150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393335e-2df6-4c73-8e3e-ee7e0ee70cff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:00.353117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393335e-2df6-4c73-8e3e-ee7e0ee70cff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:00.354553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393335e-2df6-4c73-8e3e-ee7e0ee70cff service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:00.369128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1243/2460] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:06.662629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e44e90-8b59-4354-8ddc-cbd2ed262f06 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:06.664040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b9e44e90-8b59-4354-8ddc-cbd2ed262f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:06.675887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e44e90-8b59-4354-8ddc-cbd2ed262f06 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:06.679965 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acd35c-0aa1-4469-b71e-606e2c337732 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:06.680748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acd35c-0aa1-4469-b71e-606e2c337732 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:06.681029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acd35c-0aa1-4469-b71e-606e2c337732 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:06.681346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24acd35c-0aa1-4469-b71e-606e2c337732 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:06.691510 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24acd35c-0aa1-4469-b71e-606e2c337732 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:06.693616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24acd35c-0aa1-4469-b71e-606e2c337732 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:06.703534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1218/2461] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:10.360514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:INSERT=4,keystone:SELECT=65 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:12.549771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393335e-2df6-4c73-8e3e-ee7e0ee70cff service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:12.551527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6393335e-2df6-4c73-8e3e-ee7e0ee70cff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:12.627373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393335e-2df6-4c73-8e3e-ee7e0ee70cff service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:12.633636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33667899-460d-46fa-865c-e7c3cc1a10c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:12.634303 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667899-460d-46fa-865c-e7c3cc1a10c1 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:12.634596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667899-460d-46fa-865c-e7c3cc1a10c1 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:12.634884 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33667899-460d-46fa-865c-e7c3cc1a10c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:12.636886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-33667899-460d-46fa-865c-e7c3cc1a10c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:12.650012 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33667899-460d-46fa-865c-e7c3cc1a10c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:12.651635 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33667899-460d-46fa-865c-e7c3cc1a10c1 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:12.663692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1244/2462] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:16.699176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=21 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:22.658485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=34 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:33.516289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acd35c-0aa1-4469-b71e-606e2c337732 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:33.577764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24acd35c-0aa1-4469-b71e-606e2c337732 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:33.582259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:33.583111 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:33.583391 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:33.583663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] PATH_INFO: `/v3/users/0f28795e34204832a8ac634e0ebd5aae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:33.585358 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f28795e34204832a8ac634e0ebd5aae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:33.587263 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:33.794256 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f28795e34204832a8ac634e0ebd5aae'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:33.801200 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f28795e34204832a8ac634e0ebd5aae'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:33.801998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f28795e34204832a8ac634e0ebd5aae'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:33.813764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f28795e34204832a8ac634e0ebd5aae'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:33.814412 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f28795e34204832a8ac634e0ebd5aae'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:33.900787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1219/2463] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:33 2022] DELETE /identity/v3/users/0f28795e34204832a8ac634e0ebd5aae => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:33.905345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b03e0-9cfa-41bc-a46a-5e22c5218a7b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:33.905841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b03e0-9cfa-41bc-a46a-5e22c5218a7b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:33.905937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55b03e0-9cfa-41bc-a46a-5e22c5218a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:33.923075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f55b03e0-9cfa-41bc-a46a-5e22c5218a7b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:33.924286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f55b03e0-9cfa-41bc-a46a-5e22c5218a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:33.981583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1245/2464] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:33.991215 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:33.992640 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:34.048632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa8306-d77a-4904-b13e-ad34f005a6b4 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:34.052112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd6cfa-24a0-408c-9fcf-d43015c6c7eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:34.052752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd6cfa-24a0-408c-9fcf-d43015c6c7eb service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:34.053033 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd6cfa-24a0-408c-9fcf-d43015c6c7eb service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:34.053336 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd6cfa-24a0-408c-9fcf-d43015c6c7eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:34.062154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd6cfa-24a0-408c-9fcf-d43015c6c7eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:34.063582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd6cfa-24a0-408c-9fcf-d43015c6c7eb service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:34.073346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1220/2465] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:34.474432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b03e0-9cfa-41bc-a46a-5e22c5218a7b None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:34.477204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f55b03e0-9cfa-41bc-a46a-5e22c5218a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:34.490778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55b03e0-9cfa-41bc-a46a-5e22c5218a7b None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:34.496491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:34.500164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:34.500164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:34.500164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] PATH_INFO: `/v3/projects/6e8fe952ec9c44c1a62e4921d343c5f4` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:34.501515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8fe952ec9c44c1a62e4921d343c5f4)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:34.503567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:34.609816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8fe952ec9c44c1a62e4921d343c5f4', 'request_id': 'req-ffa65e73-ca5e-4277-8a08-383408dc45a0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:34.683076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8fe952ec9c44c1a62e4921d343c5f4', 'request_id': 'req-ffa65e73-ca5e-4277-8a08-383408dc45a0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:34.688843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8fe952ec9c44c1a62e4921d343c5f4', 'request_id': 'req-ffa65e73-ca5e-4277-8a08-383408dc45a0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:34.699931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] The token cache is being invalidate because project 6e8fe952ec9c44c1a62e4921d343c5f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:34.701989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:34.701989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:34.702509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1246/2466] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:34 2022] DELETE /identity/v3/projects/6e8fe952ec9c44c1a62e4921d343c5f4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:34.706436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd6cfa-24a0-408c-9fcf-d43015c6c7eb service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:34.716173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd6cfa-24a0-408c-9fcf-d43015c6c7eb service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:34.734185 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:34.771370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e6732-5467-4a7d-803c-7fecf9b27f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:34.772246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e6732-5467-4a7d-803c-7fecf9b27f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:34.772562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e6732-5467-4a7d-803c-7fecf9b27f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:34.772961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e6732-5467-4a7d-803c-7fecf9b27f63 admin admin] PATH_INFO: `/v3/users/70abad4ef7e84cacb05f1e14ee70f4c5` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:34.774174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e6732-5467-4a7d-803c-7fecf9b27f63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70abad4ef7e84cacb05f1e14ee70f4c5)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:34.776103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e6732-5467-4a7d-803c-7fecf9b27f63 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:34.941922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-da0e6732-5467-4a7d-803c-7fecf9b27f63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70abad4ef7e84cacb05f1e14ee70f4c5'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:34.948663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-da0e6732-5467-4a7d-803c-7fecf9b27f63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70abad4ef7e84cacb05f1e14ee70f4c5'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:34.949362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-da0e6732-5467-4a7d-803c-7fecf9b27f63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70abad4ef7e84cacb05f1e14ee70f4c5'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:34.961629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-da0e6732-5467-4a7d-803c-7fecf9b27f63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70abad4ef7e84cacb05f1e14ee70f4c5'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:34.962284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-da0e6732-5467-4a7d-803c-7fecf9b27f63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70abad4ef7e84cacb05f1e14ee70f4c5'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:35.077747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1221/2467] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:34 2022] DELETE /identity/v3/users/70abad4ef7e84cacb05f1e14ee70f4c5 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:35.703169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:35.704808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:35.730124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa65e73-ca5e-4277-8a08-383408dc45a0 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:35.793616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:35.793616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:35.793616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:35.793616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] PATH_INFO: `/v3/projects/8d7cdfb8d29a446293c66ef7fbb3ed66` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:35.794146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d7cdfb8d29a446293c66ef7fbb3ed66)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:35.795909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:35.890619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:35.906360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d7cdfb8d29a446293c66ef7fbb3ed66', 'request_id': 'req-cedd67a0-1f70-4b79-880d-e199cce64b39'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:35.981578 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d7cdfb8d29a446293c66ef7fbb3ed66', 'request_id': 'req-cedd67a0-1f70-4b79-880d-e199cce64b39'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:35.987851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d7cdfb8d29a446293c66ef7fbb3ed66', 'request_id': 'req-cedd67a0-1f70-4b79-880d-e199cce64b39'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:35.999742 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] The token cache is being invalidate because project 8d7cdfb8d29a446293c66ef7fbb3ed66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:36.000032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:36.000667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cedd67a0-1f70-4b79-880d-e199cce64b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:36.002313 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1247/2468] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:35 2022] DELETE /identity/v3/projects/8d7cdfb8d29a446293c66ef7fbb3ed66 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:36.007093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759850ca-084f-4fb4-8d5d-5175405c5246 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.007591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759850ca-084f-4fb4-8d5d-5175405c5246 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.007687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759850ca-084f-4fb4-8d5d-5175405c5246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.025539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-759850ca-084f-4fb4-8d5d-5175405c5246 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:36.029037 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-759850ca-084f-4fb4-8d5d-5175405c5246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:36.083902 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1222/2469] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:36.091575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cfe55-0f89-49e4-9395-3ffe83a9c7eb None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.091868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cfe55-0f89-49e4-9395-3ffe83a9c7eb None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.092149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cfe55-0f89-49e4-9395-3ffe83a9c7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.110863 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-db3cfe55-0f89-49e4-9395-3ffe83a9c7eb None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:36.112066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-db3cfe55-0f89-49e4-9395-3ffe83a9c7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:36.120732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1248/2470] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:36.125452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759850ca-084f-4fb4-8d5d-5175405c5246 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:36.135184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759850ca-084f-4fb4-8d5d-5175405c5246 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:36.139735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf439c-a3d6-4899-b9c6-d24ee11d3444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:36.140387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf439c-a3d6-4899-b9c6-d24ee11d3444 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.143771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf439c-a3d6-4899-b9c6-d24ee11d3444 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.143771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf439c-a3d6-4899-b9c6-d24ee11d3444 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.143771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf439c-a3d6-4899-b9c6-d24ee11d3444 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:02:36.143771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf439c-a3d6-4899-b9c6-d24ee11d3444 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:36.143771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf439c-a3d6-4899-b9c6-d24ee11d3444 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:36.154948 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1223/2471] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:02:36.159196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cfe55-0f89-49e4-9395-3ffe83a9c7eb None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:36.168918 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cfe55-0f89-49e4-9395-3ffe83a9c7eb None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:36.173627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c42e41-ca3e-4907-9897-c4aa970b93e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:36.174271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c42e41-ca3e-4907-9897-c4aa970b93e9 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.174557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c42e41-ca3e-4907-9897-c4aa970b93e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.174838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c42e41-ca3e-4907-9897-c4aa970b93e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.175537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c42e41-ca3e-4907-9897-c4aa970b93e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:36.177464 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c42e41-ca3e-4907-9897-c4aa970b93e9 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:36.296073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1249/2472] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:02:36 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:36.300168 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf439c-a3d6-4899-b9c6-d24ee11d3444 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:36.310583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf439c-a3d6-4899-b9c6-d24ee11d3444 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:36.335676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:36.374476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641515f-b1c6-4e6c-8924-5e031ad045b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:36.375186 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641515f-b1c6-4e6c-8924-5e031ad045b7 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.375469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641515f-b1c6-4e6c-8924-5e031ad045b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.375747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641515f-b1c6-4e6c-8924-5e031ad045b7 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.376433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641515f-b1c6-4e6c-8924-5e031ad045b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:36.378429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641515f-b1c6-4e6c-8924-5e031ad045b7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:36.508045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1224/2473] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:02:36 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:36.512102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c42e41-ca3e-4907-9897-c4aa970b93e9 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:36.524578 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c42e41-ca3e-4907-9897-c4aa970b93e9 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:36.529087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400156a-755d-42dd-9467-9b39d6e88a5d None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.529427 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400156a-755d-42dd-9467-9b39d6e88a5d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.530550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400156a-755d-42dd-9467-9b39d6e88a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.530691 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a960c-e58e-4eb5-9f7a-8ea2e1b43b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:36.531724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a960c-e58e-4eb5-9f7a-8ea2e1b43b4f admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.532229 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a960c-e58e-4eb5-9f7a-8ea2e1b43b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.532612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a960c-e58e-4eb5-9f7a-8ea2e1b43b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.534105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a960c-e58e-4eb5-9f7a-8ea2e1b43b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:36.535352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a960c-e58e-4eb5-9f7a-8ea2e1b43b4f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:36.547753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1250/2474] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:36.550628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-5400156a-755d-42dd-9467-9b39d6e88a5d None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:36.551523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a960c-e58e-4eb5-9f7a-8ea2e1b43b4f admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:36.552050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5400156a-755d-42dd-9467-9b39d6e88a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:36.562195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a960c-e58e-4eb5-9f7a-8ea2e1b43b4f admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:36.567990 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75e891-d114-488e-a04b-dc7a091bd553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:36.568936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e891-d114-488e-a04b-dc7a091bd553 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.569366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e891-d114-488e-a04b-dc7a091bd553 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.569747 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75e891-d114-488e-a04b-dc7a091bd553 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.571135 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75e891-d114-488e-a04b-dc7a091bd553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:36.573316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75e891-d114-488e-a04b-dc7a091bd553 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:36.589337 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1251/2475] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:02:36.591212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75e891-d114-488e-a04b-dc7a091bd553 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:36.602468 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75e891-d114-488e-a04b-dc7a091bd553 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:36.607727 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0480-d818-4aca-bc41-f3e5b291900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:36.608856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0480-d818-4aca-bc41-f3e5b291900d admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.609437 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0480-d818-4aca-bc41-f3e5b291900d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.609869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa0480-d818-4aca-bc41-f3e5b291900d admin admin] PATH_INFO: `/v3/projects/e7ca231f49ed475c8e41410f82c97ff6/users/153df12785b346e6a36cbe1ebeefe1bb/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.618311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1225/2476] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:02:36.631704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa0480-d818-4aca-bc41-f3e5b291900d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ca231f49ed475c8e41410f82c97ff6, user_id=153df12785b346e6a36cbe1ebeefe1bb, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:36.634619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa0480-d818-4aca-bc41-f3e5b291900d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:36.754533 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1252/2477] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:36 2022] PUT /identity/v3/projects/e7ca231f49ed475c8e41410f82c97ff6/users/153df12785b346e6a36cbe1ebeefe1bb/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:02:36.766469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c82242-521c-45ca-93fb-471ae90afdd0 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.766721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c82242-521c-45ca-93fb-471ae90afdd0 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.767002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c82242-521c-45ca-93fb-471ae90afdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.783411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0480-d818-4aca-bc41-f3e5b291900d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:36.785386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-b5aa0480-d818-4aca-bc41-f3e5b291900d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:36.818254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-27c82242-521c-45ca-93fb-471ae90afdd0 None None] MFA Rules not processed for user `153df12785b346e6a36cbe1ebeefe1bb`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:36.823031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-27c82242-521c-45ca-93fb-471ae90afdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:36.885935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa0480-d818-4aca-bc41-f3e5b291900d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:36.903413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40505220-9ab5-42e0-a118-4225dd02d0b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:36.904206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40505220-9ab5-42e0-a118-4225dd02d0b0 service swift] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:36.904587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40505220-9ab5-42e0-a118-4225dd02d0b0 service swift] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:36.906455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40505220-9ab5-42e0-a118-4225dd02d0b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:36.921041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1226/2478] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:36.932602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c82242-521c-45ca-93fb-471ae90afdd0 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:36.960067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:36.986609 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40505220-9ab5-42e0-a118-4225dd02d0b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:36.988245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40505220-9ab5-42e0-a118-4225dd02d0b0 service swift] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:36.997730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1253/2479] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:02:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 19:02:37.008487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:37.020649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c82242-521c-45ca-93fb-471ae90afdd0 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.027622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28335f36-2e4d-4b3b-8b45-19af1f44606f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.028303 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28335f36-2e4d-4b3b-8b45-19af1f44606f admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.028589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28335f36-2e4d-4b3b-8b45-19af1f44606f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.028919 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28335f36-2e4d-4b3b-8b45-19af1f44606f admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.029862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28335f36-2e4d-4b3b-8b45-19af1f44606f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.031725 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28335f36-2e4d-4b3b-8b45-19af1f44606f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.126410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1227/2480] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:02:36 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:37.134490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40505220-9ab5-42e0-a118-4225dd02d0b0 service swift] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.147462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40505220-9ab5-42e0-a118-4225dd02d0b0 service swift] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.218484 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee524c-d0ee-4f45-a47f-04c085ca4303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.219162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee524c-d0ee-4f45-a47f-04c085ca4303 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.219451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee524c-d0ee-4f45-a47f-04c085ca4303 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.222100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee524c-d0ee-4f45-a47f-04c085ca4303 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.222100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee524c-d0ee-4f45-a47f-04c085ca4303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.225095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee524c-d0ee-4f45-a47f-04c085ca4303 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.309312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1254/2481] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:02:37 2022] POST /identity/v3/users => generated 300 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:37.312046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28335f36-2e4d-4b3b-8b45-19af1f44606f admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.325716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28335f36-2e4d-4b3b-8b45-19af1f44606f admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.331089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1da0aa-45e6-417c-b180-350b89329f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.333967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1da0aa-45e6-417c-b180-350b89329f3c admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.333967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1da0aa-45e6-417c-b180-350b89329f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.333967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1da0aa-45e6-417c-b180-350b89329f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.333967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1da0aa-45e6-417c-b180-350b89329f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.335938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1da0aa-45e6-417c-b180-350b89329f3c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.361853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1228/2482] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:37.368837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee524c-d0ee-4f45-a47f-04c085ca4303 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.382070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee524c-d0ee-4f45-a47f-04c085ca4303 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.387766 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349700be-8ec6-4be5-a492-41718a381916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.388684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349700be-8ec6-4be5-a492-41718a381916 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.388961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349700be-8ec6-4be5-a492-41718a381916 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.389263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349700be-8ec6-4be5-a492-41718a381916 admin admin] PATH_INFO: `/v3/projects/b9105cca9a5a4e0c94b4ffed4aaa5f49/users/305e78cc6a64438fbd318432d5587895/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.409621 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349700be-8ec6-4be5-a492-41718a381916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9105cca9a5a4e0c94b4ffed4aaa5f49, user_id=305e78cc6a64438fbd318432d5587895, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.413195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349700be-8ec6-4be5-a492-41718a381916 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.502462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1255/2483] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:37 2022] PUT /identity/v3/projects/b9105cca9a5a4e0c94b4ffed4aaa5f49/users/305e78cc6a64438fbd318432d5587895/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:37.508393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1da0aa-45e6-417c-b180-350b89329f3c admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.524707 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1da0aa-45e6-417c-b180-350b89329f3c admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.615755 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075950be-a6ec-447f-bf94-0903a6616018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.616858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075950be-a6ec-447f-bf94-0903a6616018 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.617155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075950be-a6ec-447f-bf94-0903a6616018 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.617485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075950be-a6ec-447f-bf94-0903a6616018 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.618213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075950be-a6ec-447f-bf94-0903a6616018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.619836 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075950be-a6ec-447f-bf94-0903a6616018 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.635418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1229/2484] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:37 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:37.639509 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349700be-8ec6-4be5-a492-41718a381916 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.650612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349700be-8ec6-4be5-a492-41718a381916 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.658113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac10078-9dcd-45b6-8e34-1705b797588a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.659310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac10078-9dcd-45b6-8e34-1705b797588a admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.659791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac10078-9dcd-45b6-8e34-1705b797588a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.660249 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac10078-9dcd-45b6-8e34-1705b797588a admin admin] PATH_INFO: `/v3/projects/b9105cca9a5a4e0c94b4ffed4aaa5f49/users/305e78cc6a64438fbd318432d5587895/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.662892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac10078-9dcd-45b6-8e34-1705b797588a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9105cca9a5a4e0c94b4ffed4aaa5f49, user_id=305e78cc6a64438fbd318432d5587895, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.667396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac10078-9dcd-45b6-8e34-1705b797588a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.681009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1256/2485] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:37 2022] PUT /identity/v3/projects/b9105cca9a5a4e0c94b4ffed4aaa5f49/users/305e78cc6a64438fbd318432d5587895/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:37.691467 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267fdf8-68d5-4d23-98e6-614a7e19660c None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.692231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267fdf8-68d5-4d23-98e6-614a7e19660c None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.692517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267fdf8-68d5-4d23-98e6-614a7e19660c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.723215 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1267fdf8-68d5-4d23-98e6-614a7e19660c None None] MFA Rules not processed for user `305e78cc6a64438fbd318432d5587895`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:37.724485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1267fdf8-68d5-4d23-98e6-614a7e19660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:37.819534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1230/2486] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:37.837829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac10078-9dcd-45b6-8e34-1705b797588a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:37.840913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3ac10078-9dcd-45b6-8e34-1705b797588a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:37.917351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac10078-9dcd-45b6-8e34-1705b797588a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:37.921422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028df1f-4af0-4770-bf2c-b243473e8a02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:37.922073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028df1f-4af0-4770-bf2c-b243473e8a02 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:37.922432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028df1f-4af0-4770-bf2c-b243473e8a02 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:37.922782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028df1f-4af0-4770-bf2c-b243473e8a02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:37.936405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8028df1f-4af0-4770-bf2c-b243473e8a02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:37.937876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8028df1f-4af0-4770-bf2c-b243473e8a02 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:37.951172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1257/2487] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:38.048681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267fdf8-68d5-4d23-98e6-614a7e19660c None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:38.117289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:38.134485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267fdf8-68d5-4d23-98e6-614a7e19660c None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:38.141501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:38.146346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:38.146346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:38.146346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] PATH_INFO: `/v3/users/153df12785b346e6a36cbe1ebeefe1bb` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:38.146346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=153df12785b346e6a36cbe1ebeefe1bb)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:38.147050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:38.356650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '153df12785b346e6a36cbe1ebeefe1bb'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:38.369866 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '153df12785b346e6a36cbe1ebeefe1bb'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:38.370662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '153df12785b346e6a36cbe1ebeefe1bb'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:38.386562 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '153df12785b346e6a36cbe1ebeefe1bb'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:38.387289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '153df12785b346e6a36cbe1ebeefe1bb'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:38.456914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1231/2488] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:38 2022] DELETE /identity/v3/users/153df12785b346e6a36cbe1ebeefe1bb => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:38.461915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8e8ca-a3b1-42c0-af37-71b284a0d477 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:38.462206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8e8ca-a3b1-42c0-af37-71b284a0d477 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:38.462489 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8e8ca-a3b1-42c0-af37-71b284a0d477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:38.495762 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-2bf8e8ca-a3b1-42c0-af37-71b284a0d477 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:38.497059 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-2bf8e8ca-a3b1-42c0-af37-71b284a0d477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:38.532097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:38.587462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1258/2489] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:38.594243 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:38.595776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:38.676991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d901a86-d730-4ed4-9ba6-485d3c2561ca admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:38.681545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28883056-c26b-4107-8fce-f85067c496b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:38.682213 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28883056-c26b-4107-8fce-f85067c496b1 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:38.682667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28883056-c26b-4107-8fce-f85067c496b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:38.682898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28883056-c26b-4107-8fce-f85067c496b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:38.697743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28883056-c26b-4107-8fce-f85067c496b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:38.699829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28883056-c26b-4107-8fce-f85067c496b1 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:38.712936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1232/2490] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:39.206522 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8e8ca-a3b1-42c0-af37-71b284a0d477 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:39.208349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-2bf8e8ca-a3b1-42c0-af37-71b284a0d477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:39.221674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8e8ca-a3b1-42c0-af37-71b284a0d477 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:39.236187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:39.237082 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:39.237399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:39.237674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] PATH_INFO: `/v3/projects/e7ca231f49ed475c8e41410f82c97ff6` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:39.240008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ca231f49ed475c8e41410f82c97ff6)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:39.242694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:39.395800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ca231f49ed475c8e41410f82c97ff6', 'request_id': 'req-a634a2b6-a587-4e2e-8c22-f67a37430a0b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.489588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ca231f49ed475c8e41410f82c97ff6', 'request_id': 'req-a634a2b6-a587-4e2e-8c22-f67a37430a0b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.496172 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ca231f49ed475c8e41410f82c97ff6', 'request_id': 'req-a634a2b6-a587-4e2e-8c22-f67a37430a0b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.515534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] The token cache is being invalidate because project e7ca231f49ed475c8e41410f82c97ff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:39.515534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.515534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.515534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1259/2491] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:39 2022] DELETE /identity/v3/projects/e7ca231f49ed475c8e41410f82c97ff6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:39.516798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28883056-c26b-4107-8fce-f85067c496b1 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:39.527928 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28883056-c26b-4107-8fce-f85067c496b1 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:39.612474 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d70bab-ffa3-4616-89d7-9c97e95ca8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:39.613298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70bab-ffa3-4616-89d7-9c97e95ca8fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:39.613820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70bab-ffa3-4616-89d7-9c97e95ca8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:39.614181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70bab-ffa3-4616-89d7-9c97e95ca8fc admin admin] PATH_INFO: `/v3/users/305e78cc6a64438fbd318432d5587895` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:39.615641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d70bab-ffa3-4616-89d7-9c97e95ca8fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=305e78cc6a64438fbd318432d5587895)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:39.617842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d70bab-ffa3-4616-89d7-9c97e95ca8fc admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:39.634247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:39.846206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f8d70bab-ffa3-4616-89d7-9c97e95ca8fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305e78cc6a64438fbd318432d5587895'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.855793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f8d70bab-ffa3-4616-89d7-9c97e95ca8fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305e78cc6a64438fbd318432d5587895'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.856548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f8d70bab-ffa3-4616-89d7-9c97e95ca8fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305e78cc6a64438fbd318432d5587895'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.868831 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f8d70bab-ffa3-4616-89d7-9c97e95ca8fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305e78cc6a64438fbd318432d5587895'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:39.882640 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f8d70bab-ffa3-4616-89d7-9c97e95ca8fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305e78cc6a64438fbd318432d5587895'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:40.185400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1233/2492] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:39 2022] DELETE /identity/v3/users/305e78cc6a64438fbd318432d5587895 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:40.836479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:40.838796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:40.861162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634a2b6-a587-4e2e-8c22-f67a37430a0b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:40.942690 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:40.943987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:40.944429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:40.945009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] PATH_INFO: `/v3/projects/b9105cca9a5a4e0c94b4ffed4aaa5f49` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:40.947452 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9105cca9a5a4e0c94b4ffed4aaa5f49)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:40.949919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:41.091163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9105cca9a5a4e0c94b4ffed4aaa5f49', 'request_id': 'req-0868957b-72e9-4174-b5ef-a34aa3637c1a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:41.094220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:41.158526 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9105cca9a5a4e0c94b4ffed4aaa5f49', 'request_id': 'req-0868957b-72e9-4174-b5ef-a34aa3637c1a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:41.165666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9105cca9a5a4e0c94b4ffed4aaa5f49', 'request_id': 'req-0868957b-72e9-4174-b5ef-a34aa3637c1a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:41.178979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] The token cache is being invalidate because project b9105cca9a5a4e0c94b4ffed4aaa5f49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:41.179268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:41.179931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0868957b-72e9-4174-b5ef-a34aa3637c1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:41.182779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1260/2493] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:40 2022] DELETE /identity/v3/projects/b9105cca9a5a4e0c94b4ffed4aaa5f49 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:41.188072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc243f2-5f83-4320-a491-1b84f2bef568 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:41.188572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc243f2-5f83-4320-a491-1b84f2bef568 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:41.188572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc243f2-5f83-4320-a491-1b84f2bef568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:41.213446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-bcc243f2-5f83-4320-a491-1b84f2bef568 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:41.214895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-bcc243f2-5f83-4320-a491-1b84f2bef568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:41.293446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1234/2494] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:41.301329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fb524-0edf-4be7-89ae-98f9116c63dd None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:41.301602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fb524-0edf-4be7-89ae-98f9116c63dd None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:41.301910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fb524-0edf-4be7-89ae-98f9116c63dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:41.332670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d05fb524-0edf-4be7-89ae-98f9116c63dd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:41.335276 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d05fb524-0edf-4be7-89ae-98f9116c63dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:41.348134 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1261/2495] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:41.352753 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc243f2-5f83-4320-a491-1b84f2bef568 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:41.371388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc243f2-5f83-4320-a491-1b84f2bef568 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:41.380517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f139-e39f-40c4-adcc-d4bda38403b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:41.382127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f139-e39f-40c4-adcc-d4bda38403b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:41.382534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f139-e39f-40c4-adcc-d4bda38403b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:41.382882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f139-e39f-40c4-adcc-d4bda38403b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:41.383682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5f139-e39f-40c4-adcc-d4bda38403b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:02:41.384066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5f139-e39f-40c4-adcc-d4bda38403b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:41.386230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5f139-e39f-40c4-adcc-d4bda38403b0 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:41.404406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1235/2496] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:02:41.408862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fb524-0edf-4be7-89ae-98f9116c63dd None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:41.424157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fb524-0edf-4be7-89ae-98f9116c63dd None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:41.429469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd47148-395d-4f19-a7ad-3edfcbc7e8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:41.430914 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd47148-395d-4f19-a7ad-3edfcbc7e8ed admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:41.431517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd47148-395d-4f19-a7ad-3edfcbc7e8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:41.432117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd47148-395d-4f19-a7ad-3edfcbc7e8ed admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:41.432828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd47148-395d-4f19-a7ad-3edfcbc7e8ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:41.434966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd47148-395d-4f19-a7ad-3edfcbc7e8ed admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:41.516595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1262/2497] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:02:41 2022] POST /identity/v3/projects => generated 401 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:02:41.522968 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f139-e39f-40c4-adcc-d4bda38403b0 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:41.533778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f139-e39f-40c4-adcc-d4bda38403b0 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:41.604295 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4885683-235c-4a0a-af20-df2efba8ba25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:41.605021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4885683-235c-4a0a-af20-df2efba8ba25 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:41.605348 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4885683-235c-4a0a-af20-df2efba8ba25 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:41.605509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4885683-235c-4a0a-af20-df2efba8ba25 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:41.606198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4885683-235c-4a0a-af20-df2efba8ba25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:41.608135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4885683-235c-4a0a-af20-df2efba8ba25 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:41.711616 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1236/2498] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:02:41 2022] POST /identity/v3/users => generated 301 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:41.715882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd47148-395d-4f19-a7ad-3edfcbc7e8ed admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:41.732213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd47148-395d-4f19-a7ad-3edfcbc7e8ed admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:41.740055 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562f1e2-2125-44db-89f6-f6365f130d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:41.741310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562f1e2-2125-44db-89f6-f6365f130d46 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:41.741607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562f1e2-2125-44db-89f6-f6365f130d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:41.741874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562f1e2-2125-44db-89f6-f6365f130d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:41.742770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562f1e2-2125-44db-89f6-f6365f130d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:41.744404 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562f1e2-2125-44db-89f6-f6365f130d46 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:41.761794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1263/2499] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:41.762639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4885683-235c-4a0a-af20-df2efba8ba25 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:41.776864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4885683-235c-4a0a-af20-df2efba8ba25 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:41.783546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d39ec8-8ea2-4a13-ad48-cc1dc40340ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:41.784517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d39ec8-8ea2-4a13-ad48-cc1dc40340ae admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:41.785010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d39ec8-8ea2-4a13-ad48-cc1dc40340ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:41.785459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d39ec8-8ea2-4a13-ad48-cc1dc40340ae admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:41.786287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d39ec8-8ea2-4a13-ad48-cc1dc40340ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:41.788091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d39ec8-8ea2-4a13-ad48-cc1dc40340ae admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:41.789833 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:41.810596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562f1e2-2125-44db-89f6-f6365f130d46 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:41.818502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1237/2500] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:02:41.825643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562f1e2-2125-44db-89f6-f6365f130d46 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:41.833195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b2c5c-f2c1-48e7-888f-61b8f679868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:41.835067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b2c5c-f2c1-48e7-888f-61b8f679868d admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:41.835722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b2c5c-f2c1-48e7-888f-61b8f679868d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:41.836224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b2c5c-f2c1-48e7-888f-61b8f679868d admin admin] PATH_INFO: `/v3/projects/5b499b486fd94f5ab2573883613bb213/users/38858357c0534cdc9c243d92f20d6e81/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:41.863307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b2c5c-f2c1-48e7-888f-61b8f679868d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b499b486fd94f5ab2573883613bb213, user_id=38858357c0534cdc9c243d92f20d6e81, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:41.863307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b2c5c-f2c1-48e7-888f-61b8f679868d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:41.947839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1264/2501] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:41 2022] PUT /identity/v3/projects/5b499b486fd94f5ab2573883613bb213/users/38858357c0534cdc9c243d92f20d6e81/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:41.959917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c53c9-aee0-45a3-8ba8-c8e6f2415a7e None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:41.965689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c53c9-aee0-45a3-8ba8-c8e6f2415a7e None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:41.965689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c53c9-aee0-45a3-8ba8-c8e6f2415a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:41.995830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-249c53c9-aee0-45a3-8ba8-c8e6f2415a7e None None] MFA Rules not processed for user `38858357c0534cdc9c243d92f20d6e81`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:41.999079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-249c53c9-aee0-45a3-8ba8-c8e6f2415a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:42.083926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1238/2502] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:42.085687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b2c5c-f2c1-48e7-888f-61b8f679868d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:42.173120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b2c5c-f2c1-48e7-888f-61b8f679868d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:42.181740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d44fd7-f9ba-4fa0-93e4-9896f8f52633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:42.182736 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44fd7-f9ba-4fa0-93e4-9896f8f52633 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.183150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44fd7-f9ba-4fa0-93e4-9896f8f52633 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.183561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d44fd7-f9ba-4fa0-93e4-9896f8f52633 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.184604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d44fd7-f9ba-4fa0-93e4-9896f8f52633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:42.187928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d44fd7-f9ba-4fa0-93e4-9896f8f52633 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:42.283782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1265/2503] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:02:42 2022] POST /identity/v3/projects => generated 401 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:42.284355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c53c9-aee0-45a3-8ba8-c8e6f2415a7e None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:42.299720 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c53c9-aee0-45a3-8ba8-c8e6f2415a7e None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:42.385894 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912bf84-e36e-4f01-b3f3-3dac45ead200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:42.386758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912bf84-e36e-4f01-b3f3-3dac45ead200 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.387167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912bf84-e36e-4f01-b3f3-3dac45ead200 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.387529 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912bf84-e36e-4f01-b3f3-3dac45ead200 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.388316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912bf84-e36e-4f01-b3f3-3dac45ead200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:42.390379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912bf84-e36e-4f01-b3f3-3dac45ead200 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:42.505297 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1239/2504] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:02:42 2022] POST /identity/v3/users => generated 307 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:42.510174 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d44fd7-f9ba-4fa0-93e4-9896f8f52633 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:42.521391 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d44fd7-f9ba-4fa0-93e4-9896f8f52633 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:42.529875 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad90e9-9c96-48ed-a1ed-21ce1845d74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:42.530833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad90e9-9c96-48ed-a1ed-21ce1845d74e admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.531138 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad90e9-9c96-48ed-a1ed-21ce1845d74e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.531413 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad90e9-9c96-48ed-a1ed-21ce1845d74e admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.532257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad90e9-9c96-48ed-a1ed-21ce1845d74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:42.533925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad90e9-9c96-48ed-a1ed-21ce1845d74e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:42.547398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1266/2505] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:42.552886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912bf84-e36e-4f01-b3f3-3dac45ead200 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:42.564851 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912bf84-e36e-4f01-b3f3-3dac45ead200 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:42.571741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627e72c-257e-4752-85e3-f585e2d42215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:42.572670 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627e72c-257e-4752-85e3-f585e2d42215 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.572953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627e72c-257e-4752-85e3-f585e2d42215 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.573225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627e72c-257e-4752-85e3-f585e2d42215 admin admin] PATH_INFO: `/v3/projects/7f678bf800a04c2285fc925a431b7d46/users/11f63795fa36442982c723fb998b7995/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.594217 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627e72c-257e-4752-85e3-f585e2d42215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f678bf800a04c2285fc925a431b7d46, user_id=11f63795fa36442982c723fb998b7995, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:42.596900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627e72c-257e-4752-85e3-f585e2d42215 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:42.679320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1240/2506] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:42 2022] PUT /identity/v3/projects/7f678bf800a04c2285fc925a431b7d46/users/11f63795fa36442982c723fb998b7995/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:42.683097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad90e9-9c96-48ed-a1ed-21ce1845d74e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:42.694565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad90e9-9c96-48ed-a1ed-21ce1845d74e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:42.763399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:42.780839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e331af1-b2e3-4d2c-8837-9eb747c9bb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:42.781666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e331af1-b2e3-4d2c-8837-9eb747c9bb10 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.782017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e331af1-b2e3-4d2c-8837-9eb747c9bb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.782298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e331af1-b2e3-4d2c-8837-9eb747c9bb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.783057 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e331af1-b2e3-4d2c-8837-9eb747c9bb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:42.785165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e331af1-b2e3-4d2c-8837-9eb747c9bb10 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:42.797195 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1267/2507] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:42.811126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627e72c-257e-4752-85e3-f585e2d42215 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:42.823398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627e72c-257e-4752-85e3-f585e2d42215 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:42.829025 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcda48-5c7e-4b73-b7c9-89cd1cfa3118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:42.830147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcda48-5c7e-4b73-b7c9-89cd1cfa3118 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.830436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcda48-5c7e-4b73-b7c9-89cd1cfa3118 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.830713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fcda48-5c7e-4b73-b7c9-89cd1cfa3118 admin admin] PATH_INFO: `/v3/projects/7f678bf800a04c2285fc925a431b7d46/users/11f63795fa36442982c723fb998b7995/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.832877 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcda48-5c7e-4b73-b7c9-89cd1cfa3118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f678bf800a04c2285fc925a431b7d46, user_id=11f63795fa36442982c723fb998b7995, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:42.836359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fcda48-5c7e-4b73-b7c9-89cd1cfa3118 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:42.850175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1241/2508] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:42 2022] PUT /identity/v3/projects/7f678bf800a04c2285fc925a431b7d46/users/11f63795fa36442982c723fb998b7995/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:42.860363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864d83a-2b74-490b-a605-26af40eb5f5f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:42.860651 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864d83a-2b74-490b-a605-26af40eb5f5f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:42.860935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864d83a-2b74-490b-a605-26af40eb5f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:42.888132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d864d83a-2b74-490b-a605-26af40eb5f5f None None] MFA Rules not processed for user `11f63795fa36442982c723fb998b7995`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:42.889433 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d864d83a-2b74-490b-a605-26af40eb5f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:42.970141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1268/2509] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:42 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:42.981402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcda48-5c7e-4b73-b7c9-89cd1cfa3118 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:42.983151 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a8fcda48-5c7e-4b73-b7c9-89cd1cfa3118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:43.037555 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:43.056662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fcda48-5c7e-4b73-b7c9-89cd1cfa3118 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:43.061066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8306bb0-4d01-47d7-aeb7-b77ec3acd206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:43.061800 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8306bb0-4d01-47d7-aeb7-b77ec3acd206 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:43.062085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8306bb0-4d01-47d7-aeb7-b77ec3acd206 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:43.062433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8306bb0-4d01-47d7-aeb7-b77ec3acd206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:43.073141 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8306bb0-4d01-47d7-aeb7-b77ec3acd206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:43.074816 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8306bb0-4d01-47d7-aeb7-b77ec3acd206 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:43.086541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1242/2510] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:42 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:43.333008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864d83a-2b74-490b-a605-26af40eb5f5f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:43.355710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864d83a-2b74-490b-a605-26af40eb5f5f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:43.364399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d8ea1-b86e-41ef-a068-0d7f32ce9b58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:43.365131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d8ea1-b86e-41ef-a068-0d7f32ce9b58 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:43.365477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d8ea1-b86e-41ef-a068-0d7f32ce9b58 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:43.366108 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d8ea1-b86e-41ef-a068-0d7f32ce9b58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:43.441435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d8ea1-b86e-41ef-a068-0d7f32ce9b58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:43.442982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d8ea1-b86e-41ef-a068-0d7f32ce9b58 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:43.452858 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1269/2511] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:48.535707 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8306bb0-4d01-47d7-aeb7-b77ec3acd206 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:48.601273 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8306bb0-4d01-47d7-aeb7-b77ec3acd206 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:48.606512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:48.607587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:48.609589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:48.609589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] PATH_INFO: `/v3/users/38858357c0534cdc9c243d92f20d6e81` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:48.609589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38858357c0534cdc9c243d92f20d6e81)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:48.611837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:48.798243 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38858357c0534cdc9c243d92f20d6e81'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:48.805772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38858357c0534cdc9c243d92f20d6e81'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:48.806456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38858357c0534cdc9c243d92f20d6e81'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:48.819460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38858357c0534cdc9c243d92f20d6e81'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:48.820294 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38858357c0534cdc9c243d92f20d6e81'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:48.883973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1243/2512] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:48 2022] DELETE /identity/v3/users/38858357c0534cdc9c243d92f20d6e81 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:48.888448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4dee4-0999-4a40-9e84-b4b43655cfba None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:48.888909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4dee4-0999-4a40-9e84-b4b43655cfba None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:48.889003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4dee4-0999-4a40-9e84-b4b43655cfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:48.906994 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-5ed4dee4-0999-4a40-9e84-b4b43655cfba None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:48.908206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5ed4dee4-0999-4a40-9e84-b4b43655cfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:48.972331 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1270/2513] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:48.997166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:48.998718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:49.057438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2aabf-cef0-4bc3-9348-03fe71683b12 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:49.061093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b78746-88df-4708-af32-8fc0a6888457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:49.061787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b78746-88df-4708-af32-8fc0a6888457 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:49.064832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b78746-88df-4708-af32-8fc0a6888457 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:49.064832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b78746-88df-4708-af32-8fc0a6888457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:49.072359 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b78746-88df-4708-af32-8fc0a6888457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:49.073858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b78746-88df-4708-af32-8fc0a6888457 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:49.084245 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1244/2514] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:49.463253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4dee4-0999-4a40-9e84-b4b43655cfba None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:49.468488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5ed4dee4-0999-4a40-9e84-b4b43655cfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:49.479761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4dee4-0999-4a40-9e84-b4b43655cfba None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:49.484952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:49.486044 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:49.486302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:49.486579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] PATH_INFO: `/v3/projects/5b499b486fd94f5ab2573883613bb213` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:49.487764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b499b486fd94f5ab2573883613bb213)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:49.489710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:49.497007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:49.575794 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b499b486fd94f5ab2573883613bb213', 'request_id': 'req-832f833c-2180-49e9-93a4-a82676288aea'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:49.615923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b499b486fd94f5ab2573883613bb213', 'request_id': 'req-832f833c-2180-49e9-93a4-a82676288aea'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:49.622163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b499b486fd94f5ab2573883613bb213', 'request_id': 'req-832f833c-2180-49e9-93a4-a82676288aea'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:49.633480 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] The token cache is being invalidate because project 5b499b486fd94f5ab2573883613bb213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:49.633764 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:49.634389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:49.635913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1271/2515] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:49 2022] DELETE /identity/v3/projects/5b499b486fd94f5ab2573883613bb213 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:49.639843 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b78746-88df-4708-af32-8fc0a6888457 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:49.644664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:49.649831 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b78746-88df-4708-af32-8fc0a6888457 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:49.704608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501eda5-f096-48f3-ad1c-7f06e9bd5646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:49.705507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501eda5-f096-48f3-ad1c-7f06e9bd5646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:49.705794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501eda5-f096-48f3-ad1c-7f06e9bd5646 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:49.706072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501eda5-f096-48f3-ad1c-7f06e9bd5646 admin admin] PATH_INFO: `/v3/users/11f63795fa36442982c723fb998b7995` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:49.707241 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501eda5-f096-48f3-ad1c-7f06e9bd5646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f63795fa36442982c723fb998b7995)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:49.709162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501eda5-f096-48f3-ad1c-7f06e9bd5646 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:49.897065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f501eda5-f096-48f3-ad1c-7f06e9bd5646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f63795fa36442982c723fb998b7995'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:49.904376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f501eda5-f096-48f3-ad1c-7f06e9bd5646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f63795fa36442982c723fb998b7995'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:49.905026 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f501eda5-f096-48f3-ad1c-7f06e9bd5646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f63795fa36442982c723fb998b7995'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:49.917191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f501eda5-f096-48f3-ad1c-7f06e9bd5646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f63795fa36442982c723fb998b7995'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:49.917844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-f501eda5-f096-48f3-ad1c-7f06e9bd5646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f63795fa36442982c723fb998b7995'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:49.986827 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1245/2516] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:02:49 2022] DELETE /identity/v3/users/11f63795fa36442982c723fb998b7995 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:50.366211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:50.368004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:50.383201 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f833c-2180-49e9-93a4-a82676288aea admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:50.441603 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:50.442462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:50.442865 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:50.443145 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] PATH_INFO: `/v3/projects/7f678bf800a04c2285fc925a431b7d46` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:50.444295 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f678bf800a04c2285fc925a431b7d46)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:50.446226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:50.558244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f678bf800a04c2285fc925a431b7d46', 'request_id': 'req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:50.615808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f678bf800a04c2285fc925a431b7d46', 'request_id': 'req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:50.622178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f678bf800a04c2285fc925a431b7d46', 'request_id': 'req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:50.633485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] The token cache is being invalidate because project 7f678bf800a04c2285fc925a431b7d46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:02:50.633776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:50.635043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f5f5d0bd-9a88-4f67-9b85-0cd0cb152d27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:02:50.636607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1272/2517] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:02:50 2022] DELETE /identity/v3/projects/7f678bf800a04c2285fc925a431b7d46 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:50.642564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1903077-c99a-4fa2-9d49-5618d653b4d4 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:50.643073 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1903077-c99a-4fa2-9d49-5618d653b4d4 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:50.643172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1903077-c99a-4fa2-9d49-5618d653b4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:50.661395 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b1903077-c99a-4fa2-9d49-5618d653b4d4 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:50.662603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b1903077-c99a-4fa2-9d49-5618d653b4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:50.726946 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1246/2518] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:50.735006 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0caf7a-a755-4dd5-b89e-7daf0c24cfb6 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:50.743086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0caf7a-a755-4dd5-b89e-7daf0c24cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:50.743086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0caf7a-a755-4dd5-b89e-7daf0c24cfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:50.754100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ad0caf7a-a755-4dd5-b89e-7daf0c24cfb6 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:50.755302 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ad0caf7a-a755-4dd5-b89e-7daf0c24cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:50.763757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1273/2519] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:50.770048 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1903077-c99a-4fa2-9d49-5618d653b4d4 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:50.777384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1903077-c99a-4fa2-9d49-5618d653b4d4 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:50.781954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28072a31-e33c-4e75-8267-5d256030060a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:50.782602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28072a31-e33c-4e75-8267-5d256030060a admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:50.782890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28072a31-e33c-4e75-8267-5d256030060a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:50.783164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28072a31-e33c-4e75-8267-5d256030060a admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:50.783889 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28072a31-e33c-4e75-8267-5d256030060a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:02:50.784174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28072a31-e33c-4e75-8267-5d256030060a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:50.785808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28072a31-e33c-4e75-8267-5d256030060a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:50.797668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1247/2520] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:02:50.801959 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0caf7a-a755-4dd5-b89e-7daf0c24cfb6 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:50.811664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0caf7a-a755-4dd5-b89e-7daf0c24cfb6 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:50.816300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b549c62-8a2d-4f2b-a938-3577e271db32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:50.816939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b549c62-8a2d-4f2b-a938-3577e271db32 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:50.817223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b549c62-8a2d-4f2b-a938-3577e271db32 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:50.817534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b549c62-8a2d-4f2b-a938-3577e271db32 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:50.818237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b549c62-8a2d-4f2b-a938-3577e271db32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:50.820122 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b549c62-8a2d-4f2b-a938-3577e271db32 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:50.834322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:50.912663 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1274/2521] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:02:50 2022] POST /identity/v3/projects => generated 361 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:02:50.913959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28072a31-e33c-4e75-8267-5d256030060a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:50.931889 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28072a31-e33c-4e75-8267-5d256030060a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:50.931889 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:50.981541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b958c4f-0709-4469-81ca-3fbf6bf9cea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:50.982226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958c4f-0709-4469-81ca-3fbf6bf9cea3 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:50.982727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958c4f-0709-4469-81ca-3fbf6bf9cea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:50.983035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b958c4f-0709-4469-81ca-3fbf6bf9cea3 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:50.984132 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b958c4f-0709-4469-81ca-3fbf6bf9cea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:50.986144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b958c4f-0709-4469-81ca-3fbf6bf9cea3 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:51.091147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1248/2522] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:02:50 2022] POST /identity/v3/users => generated 281 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:02:51.095282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b549c62-8a2d-4f2b-a938-3577e271db32 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:51.105984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b549c62-8a2d-4f2b-a938-3577e271db32 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:51.111008 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921a84b-e110-4441-bf93-480eef2d29b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:51.111667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921a84b-e110-4441-bf93-480eef2d29b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:51.111979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921a84b-e110-4441-bf93-480eef2d29b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:51.119144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921a84b-e110-4441-bf93-480eef2d29b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:51.119144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921a84b-e110-4441-bf93-480eef2d29b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:51.119144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921a84b-e110-4441-bf93-480eef2d29b0 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:51.132045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1275/2523] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:51.133483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b958c4f-0709-4469-81ca-3fbf6bf9cea3 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:51.143781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b958c4f-0709-4469-81ca-3fbf6bf9cea3 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:51.148493 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ced272-206e-4f6f-8ebf-b45aa04b8532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:51.149182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ced272-206e-4f6f-8ebf-b45aa04b8532 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:51.149503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ced272-206e-4f6f-8ebf-b45aa04b8532 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:51.149779 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ced272-206e-4f6f-8ebf-b45aa04b8532 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:51.150492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ced272-206e-4f6f-8ebf-b45aa04b8532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:51.152113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ced272-206e-4f6f-8ebf-b45aa04b8532 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:51.162099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1249/2524] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:02:51.166909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921a84b-e110-4441-bf93-480eef2d29b0 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:51.177023 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921a84b-e110-4441-bf93-480eef2d29b0 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:51.182514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a695dce-f30d-4715-b7a6-cfff8b64ea58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:51.183310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a695dce-f30d-4715-b7a6-cfff8b64ea58 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:51.183763 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a695dce-f30d-4715-b7a6-cfff8b64ea58 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:51.183885 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a695dce-f30d-4715-b7a6-cfff8b64ea58 admin admin] PATH_INFO: `/v3/projects/9e69d72ef4584bf5872a2512035dfe11/users/b94704a420db42108fb35089605cbe22/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:51.198937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a695dce-f30d-4715-b7a6-cfff8b64ea58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e69d72ef4584bf5872a2512035dfe11, user_id=b94704a420db42108fb35089605cbe22, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:51.201598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a695dce-f30d-4715-b7a6-cfff8b64ea58 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:51.462851 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1276/2525] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:02:51 2022] PUT /identity/v3/projects/9e69d72ef4584bf5872a2512035dfe11/users/b94704a420db42108fb35089605cbe22/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:02:51.474042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5c876-7c73-4c74-9f2e-6986291051f6 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:51.474333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5c876-7c73-4c74-9f2e-6986291051f6 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:51.474617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5c876-7c73-4c74-9f2e-6986291051f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:51.509005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-5ab5c876-7c73-4c74-9f2e-6986291051f6 None None] MFA Rules not processed for user `b94704a420db42108fb35089605cbe22`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:51.510268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5ab5c876-7c73-4c74-9f2e-6986291051f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:51.578096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1250/2526] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:02:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:51.589782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a695dce-f30d-4715-b7a6-cfff8b64ea58 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:51.591460 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0a695dce-f30d-4715-b7a6-cfff8b64ea58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:51.650611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a695dce-f30d-4715-b7a6-cfff8b64ea58 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:51.654224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9e12c-0a5c-44a7-9699-1f864be41081 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:51.654892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9e12c-0a5c-44a7-9699-1f864be41081 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:51.655175 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9e12c-0a5c-44a7-9699-1f864be41081 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:51.655446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9e12c-0a5c-44a7-9699-1f864be41081 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:51.667407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9e12c-0a5c-44a7-9699-1f864be41081 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:51.667407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9e12c-0a5c-44a7-9699-1f864be41081 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:51.674553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1277/2527] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:02:53.382766 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b3968-bedf-4690-b244-a1e2d63e5282 None None] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:53.389969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b3968-bedf-4690-b244-a1e2d63e5282 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:53.389969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4b3968-bedf-4690-b244-a1e2d63e5282 None None] PATH_INFO: `/` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:53.389969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1251/2528] 10.222.0.81 () {54 vars in 921 bytes} [Thu Jun 16 19:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 19:02:53.390573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f736a-41ae-4e84-a0c5-fe40ba533bbf None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:53.391362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f736a-41ae-4e84-a0c5-fe40ba533bbf None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:53.391900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f736a-41ae-4e84-a0c5-fe40ba533bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:53.411869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d78f736a-41ae-4e84-a0c5-fe40ba533bbf None None] MFA Rules not processed for user `251ed026ce8046e98188066bf0bc40b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:53.413396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d78f736a-41ae-4e84-a0c5-fe40ba533bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:53.496446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1278/2529] 10.222.0.81 () {58 vars in 1018 bytes} [Thu Jun 16 19:02:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:53.510089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b3968-bedf-4690-b244-a1e2d63e5282 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:53.514687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6f4b3968-bedf-4690-b244-a1e2d63e5282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:53.569894 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4b3968-bedf-4690-b244-a1e2d63e5282 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:53.573989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da0665-595d-4760-9d5e-fb269beae054 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17b7a7128d64b2ba548add02d3ec264', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:53.574627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da0665-595d-4760-9d5e-fb269beae054 service glance] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:53.574961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da0665-595d-4760-9d5e-fb269beae054 service glance] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:53.575166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da0665-595d-4760-9d5e-fb269beae054 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:53.592922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da0665-595d-4760-9d5e-fb269beae054 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:53.592922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da0665-595d-4760-9d5e-fb269beae054 service glance] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:53.609171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1252/2530] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:02:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:02:57.845255 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7445c-5c6a-4f39-b1fe-46a3e7bef1a9 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:57.845760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7445c-5c6a-4f39-b1fe-46a3e7bef1a9 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:57.845874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7445c-5c6a-4f39-b1fe-46a3e7bef1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:57.864205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d5e7445c-5c6a-4f39-b1fe-46a3e7bef1a9 None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:02:57.865502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d5e7445c-5c6a-4f39-b1fe-46a3e7bef1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:57.917067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:57.924978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1279/2531] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 19:02:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:02:57.937292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da0665-595d-4760-9d5e-fb269beae054 service glance] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:02:57.938793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-83da0665-595d-4760-9d5e-fb269beae054 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:02:57.967849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:02:58.025351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da0665-595d-4760-9d5e-fb269beae054 service glance] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:02:58.031475 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4c24a-5f7e-45b2-9cc2-5c15b686e55f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:02:58.032459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4c24a-5f7e-45b2-9cc2-5c15b686e55f service swift] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:02:58.033043 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4c24a-5f7e-45b2-9cc2-5c15b686e55f service swift] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:02:58.033353 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4c24a-5f7e-45b2-9cc2-5c15b686e55f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:02:58.046615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4c24a-5f7e-45b2-9cc2-5c15b686e55f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:02:58.048471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4c24a-5f7e-45b2-9cc2-5c15b686e55f service swift] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:02:58.068196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1253/2532] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:02:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:03:06.669522 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7445c-5c6a-4f39-b1fe-46a3e7bef1a9 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:06.670966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d5e7445c-5c6a-4f39-b1fe-46a3e7bef1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:06.730268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7445c-5c6a-4f39-b1fe-46a3e7bef1a9 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:06.733796 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccf2f6-9581-4a63-9d5b-400c68299d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:06.734470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccf2f6-9581-4a63-9d5b-400c68299d55 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:06.734760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccf2f6-9581-4a63-9d5b-400c68299d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:06.735033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccf2f6-9581-4a63-9d5b-400c68299d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:06.736965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8fccf2f6-9581-4a63-9d5b-400c68299d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:06.801724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccf2f6-9581-4a63-9d5b-400c68299d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:06.803153 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccf2f6-9581-4a63-9d5b-400c68299d55 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:06.812545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1280/2533] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:08.200399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=23 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:11.629983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4c24a-5f7e-45b2-9cc2-5c15b686e55f service swift] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:11.634192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-3ec4c24a-5f7e-45b2-9cc2-5c15b686e55f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:11.703661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4c24a-5f7e-45b2-9cc2-5c15b686e55f service swift] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:11.708941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05d1cd-3986-4b70-ba8e-ae06a52d7243 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:11.709642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05d1cd-3986-4b70-ba8e-ae06a52d7243 service placement] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:11.709931 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05d1cd-3986-4b70-ba8e-ae06a52d7243 service placement] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:11.710209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b05d1cd-3986-4b70-ba8e-ae06a52d7243 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:11.711978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1b05d1cd-3986-4b70-ba8e-ae06a52d7243 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:11.724607 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05d1cd-3986-4b70-ba8e-ae06a52d7243 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:11.731284 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b05d1cd-3986-4b70-ba8e-ae06a52d7243 service placement] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:11.736670 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1254/2534] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:11.762032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccf2f6-9581-4a63-9d5b-400c68299d55 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:11.763631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8fccf2f6-9581-4a63-9d5b-400c68299d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:11.777921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccf2f6-9581-4a63-9d5b-400c68299d55 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:11.789674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04e594-03df-4198-b253-919e6a9cec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:11.790553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04e594-03df-4198-b253-919e6a9cec4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:11.790842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04e594-03df-4198-b253-919e6a9cec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:11.791123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04e594-03df-4198-b253-919e6a9cec4b admin admin] PATH_INFO: `/v3/users/fec4e3ac2e44403b9d45b37b5a6f64a9` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:11.792979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d04e594-03df-4198-b253-919e6a9cec4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fec4e3ac2e44403b9d45b37b5a6f64a9)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:11.794950 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d04e594-03df-4198-b253-919e6a9cec4b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:11.984977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7d04e594-03df-4198-b253-919e6a9cec4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fec4e3ac2e44403b9d45b37b5a6f64a9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:11.985937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7d04e594-03df-4198-b253-919e6a9cec4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fec4e3ac2e44403b9d45b37b5a6f64a9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:11.996250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7d04e594-03df-4198-b253-919e6a9cec4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fec4e3ac2e44403b9d45b37b5a6f64a9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:11.997257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7d04e594-03df-4198-b253-919e6a9cec4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fec4e3ac2e44403b9d45b37b5a6f64a9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:12.017641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7d04e594-03df-4198-b253-919e6a9cec4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fec4e3ac2e44403b9d45b37b5a6f64a9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:12.116649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1281/2535] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:11 2022] DELETE /identity/v3/users/fec4e3ac2e44403b9d45b37b5a6f64a9 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:12.391366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05d1cd-3986-4b70-ba8e-ae06a52d7243 service placement] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:12.425084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b05d1cd-3986-4b70-ba8e-ae06a52d7243 service placement] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:12.483496 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:12.484544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:12.484859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:12.485161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] PATH_INFO: `/v3/projects/f9e5f71068aa4d75bcc4578b741f490d` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:12.486403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9e5f71068aa4d75bcc4578b741f490d)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:12.488312 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:12.590426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9e5f71068aa4d75bcc4578b741f490d', 'request_id': 'req-95db8c9e-990a-4819-b6a3-45acbd2abdc3'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:12.663254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9e5f71068aa4d75bcc4578b741f490d', 'request_id': 'req-95db8c9e-990a-4819-b6a3-45acbd2abdc3'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:12.669943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9e5f71068aa4d75bcc4578b741f490d', 'request_id': 'req-95db8c9e-990a-4819-b6a3-45acbd2abdc3'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:12.682941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] The token cache is being invalidate because project f9e5f71068aa4d75bcc4578b741f490d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:12.683228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:12.683910 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-95db8c9e-990a-4819-b6a3-45acbd2abdc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:12.685864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1255/2536] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:12 2022] DELETE /identity/v3/projects/f9e5f71068aa4d75bcc4578b741f490d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:12.692575 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142528-e6d3-40b6-aada-091d181283b5 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:12.693058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142528-e6d3-40b6-aada-091d181283b5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:12.693393 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a142528-e6d3-40b6-aada-091d181283b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:12.712057 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-1a142528-e6d3-40b6-aada-091d181283b5 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:12.713307 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-1a142528-e6d3-40b6-aada-091d181283b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:12.748099 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:12.790015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1282/2537] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:03:12.798633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142cdda-e560-49eb-83f5-66f011d906d8 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:12.798928 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142cdda-e560-49eb-83f5-66f011d906d8 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:12.799270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142cdda-e560-49eb-83f5-66f011d906d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:12.818124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1142cdda-e560-49eb-83f5-66f011d906d8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:12.819339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1142cdda-e560-49eb-83f5-66f011d906d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:12.827319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1256/2538] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:12.831692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a142528-e6d3-40b6-aada-091d181283b5 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:12.845917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a142528-e6d3-40b6-aada-091d181283b5 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:12.846180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa32b1c-c41c-4062-a5ca-0c1eaa370cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:12.846842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32b1c-c41c-4062-a5ca-0c1eaa370cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:12.847126 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32b1c-c41c-4062-a5ca-0c1eaa370cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:12.847400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32b1c-c41c-4062-a5ca-0c1eaa370cf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:12.848118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa32b1c-c41c-4062-a5ca-0c1eaa370cf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:03:12.848427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa32b1c-c41c-4062-a5ca-0c1eaa370cf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:12.850084 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa32b1c-c41c-4062-a5ca-0c1eaa370cf8 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:12.862900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1283/2539] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:03:12.867389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142cdda-e560-49eb-83f5-66f011d906d8 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:12.878161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142cdda-e560-49eb-83f5-66f011d906d8 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:12.882839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3975b07-6dbf-48ad-9174-1a7a144f352d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:12.883491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3975b07-6dbf-48ad-9174-1a7a144f352d admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:12.883778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3975b07-6dbf-48ad-9174-1a7a144f352d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:12.884059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3975b07-6dbf-48ad-9174-1a7a144f352d admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:12.884756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3975b07-6dbf-48ad-9174-1a7a144f352d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:12.886733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3975b07-6dbf-48ad-9174-1a7a144f352d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:12.896953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:12.968268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1257/2540] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:12 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:12.972553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa32b1c-c41c-4062-a5ca-0c1eaa370cf8 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:12.983657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa32b1c-c41c-4062-a5ca-0c1eaa370cf8 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:13.052458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a336e2-65a1-47f6-a5c1-7d1bb0cb0fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:13.053252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a336e2-65a1-47f6-a5c1-7d1bb0cb0fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:13.053561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a336e2-65a1-47f6-a5c1-7d1bb0cb0fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:13.053841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a336e2-65a1-47f6-a5c1-7d1bb0cb0fc9 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:13.054636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a336e2-65a1-47f6-a5c1-7d1bb0cb0fc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:13.056670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a336e2-65a1-47f6-a5c1-7d1bb0cb0fc9 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:13.161553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1284/2541] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:12 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:13.165643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3975b07-6dbf-48ad-9174-1a7a144f352d admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:13.176770 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3975b07-6dbf-48ad-9174-1a7a144f352d admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:13.182999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fec7f-63eb-435f-aed7-f09c81743b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:13.182999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fec7f-63eb-435f-aed7-f09c81743b01 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:13.182999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fec7f-63eb-435f-aed7-f09c81743b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:13.183405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fec7f-63eb-435f-aed7-f09c81743b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:13.184121 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fec7f-63eb-435f-aed7-f09c81743b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:13.185774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fec7f-63eb-435f-aed7-f09c81743b01 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:13.206052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1258/2542] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:13.210106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a336e2-65a1-47f6-a5c1-7d1bb0cb0fc9 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:13.221982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a336e2-65a1-47f6-a5c1-7d1bb0cb0fc9 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:13.227373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362cce9-578b-4998-a9fb-4ba388b886fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:13.228038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362cce9-578b-4998-a9fb-4ba388b886fb admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:13.228320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362cce9-578b-4998-a9fb-4ba388b886fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:13.228594 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362cce9-578b-4998-a9fb-4ba388b886fb admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:13.229962 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362cce9-578b-4998-a9fb-4ba388b886fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:13.231534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362cce9-578b-4998-a9fb-4ba388b886fb admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:13.242230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1285/2543] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:13.249164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fec7f-63eb-435f-aed7-f09c81743b01 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:13.259775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fec7f-63eb-435f-aed7-f09c81743b01 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:13.264394 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc07029e-350f-4f10-900b-8365c4a95ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:13.265302 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07029e-350f-4f10-900b-8365c4a95ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:13.265596 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07029e-350f-4f10-900b-8365c4a95ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:13.265876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc07029e-350f-4f10-900b-8365c4a95ee2 admin admin] PATH_INFO: `/v3/projects/fd982842e23d4e95975242444172c788/users/b4fb7ebe1709482aa255346ecd1d1d7d/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:13.281203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07029e-350f-4f10-900b-8365c4a95ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd982842e23d4e95975242444172c788, user_id=b4fb7ebe1709482aa255346ecd1d1d7d, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:13.283940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc07029e-350f-4f10-900b-8365c4a95ee2 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:13.356046 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1259/2544] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:13 2022] PUT /identity/v3/projects/fd982842e23d4e95975242444172c788/users/b4fb7ebe1709482aa255346ecd1d1d7d/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:13.367875 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c1018-6705-403c-9fd3-a8c80630af55 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:13.368169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c1018-6705-403c-9fd3-a8c80630af55 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:13.368472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c1018-6705-403c-9fd3-a8c80630af55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:13.396825 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6f8c1018-6705-403c-9fd3-a8c80630af55 None None] MFA Rules not processed for user `b4fb7ebe1709482aa255346ecd1d1d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:13.398149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6f8c1018-6705-403c-9fd3-a8c80630af55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:13.421335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c86af-61ae-40a7-9334-934638cae2c5 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:13.421630 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c86af-61ae-40a7-9334-934638cae2c5 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:13.421915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c86af-61ae-40a7-9334-934638cae2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:13.441308 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d14c86af-61ae-40a7-9334-934638cae2c5 None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:13.442561 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d14c86af-61ae-40a7-9334-934638cae2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:13.486308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1286/2545] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:13.499290 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c1018-6705-403c-9fd3-a8c80630af55 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:13.500832 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6f8c1018-6705-403c-9fd3-a8c80630af55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:13.515812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1260/2546] 10.222.0.81 () {58 vars in 967 bytes} [Thu Jun 16 19:03:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:03:13.522642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:13.529288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c86af-61ae-40a7-9334-934638cae2c5 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:13.530716 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d14c86af-61ae-40a7-9334-934638cae2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:13.593222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c1018-6705-403c-9fd3-a8c80630af55 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:13.599215 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5573a-8dc3-4dae-85a8-39c6da0f85bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:13.599870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5573a-8dc3-4dae-85a8-39c6da0f85bb service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:13.600162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5573a-8dc3-4dae-85a8-39c6da0f85bb service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:13.600446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5573a-8dc3-4dae-85a8-39c6da0f85bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:13.609596 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5573a-8dc3-4dae-85a8-39c6da0f85bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:13.611209 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5573a-8dc3-4dae-85a8-39c6da0f85bb service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:13.617102 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c86af-61ae-40a7-9334-934638cae2c5 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:13.620680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1287/2547] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:03:13.629555 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40996f-0bc9-4529-a047-79620d828be4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:13.630221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40996f-0bc9-4529-a047-79620d828be4 service swift] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:13.630508 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40996f-0bc9-4529-a047-79620d828be4 service swift] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:13.630788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40996f-0bc9-4529-a047-79620d828be4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:13.640374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40996f-0bc9-4529-a047-79620d828be4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:13.642298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40996f-0bc9-4529-a047-79620d828be4 service swift] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:13.651408 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1261/2548] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:03:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 19:03:13.857935 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5573a-8dc3-4dae-85a8-39c6da0f85bb service nova] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:13.922824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5573a-8dc3-4dae-85a8-39c6da0f85bb service nova] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:13.927353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0dcda-fe0a-4bbe-93e3-250ea2e49f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:13.928224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0dcda-fe0a-4bbe-93e3-250ea2e49f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:13.928507 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0dcda-fe0a-4bbe-93e3-250ea2e49f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:13.928783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0dcda-fe0a-4bbe-93e3-250ea2e49f5a admin admin] PATH_INFO: `/v3/users/b4fb7ebe1709482aa255346ecd1d1d7d` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:13.929984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0dcda-fe0a-4bbe-93e3-250ea2e49f5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4fb7ebe1709482aa255346ecd1d1d7d)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:13.937064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0dcda-fe0a-4bbe-93e3-250ea2e49f5a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:14.130542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-76e0dcda-fe0a-4bbe-93e3-250ea2e49f5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4fb7ebe1709482aa255346ecd1d1d7d'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:14.131225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-76e0dcda-fe0a-4bbe-93e3-250ea2e49f5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4fb7ebe1709482aa255346ecd1d1d7d'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:14.140211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-76e0dcda-fe0a-4bbe-93e3-250ea2e49f5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4fb7ebe1709482aa255346ecd1d1d7d'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:14.141405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-76e0dcda-fe0a-4bbe-93e3-250ea2e49f5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4fb7ebe1709482aa255346ecd1d1d7d'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:14.154974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-76e0dcda-fe0a-4bbe-93e3-250ea2e49f5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4fb7ebe1709482aa255346ecd1d1d7d'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:14.173126 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40996f-0bc9-4529-a047-79620d828be4 service swift] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:14.174573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-dc40996f-0bc9-4529-a047-79620d828be4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:14.239854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1288/2549] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:13 2022] DELETE /identity/v3/users/b4fb7ebe1709482aa255346ecd1d1d7d => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:14.242981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40996f-0bc9-4529-a047-79620d828be4 service swift] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:14.242981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:14.243225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61185ebb-b275-486e-8fa6-b757b1054ab8 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:14.243225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61185ebb-b275-486e-8fa6-b757b1054ab8 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:14.243225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61185ebb-b275-486e-8fa6-b757b1054ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:14.266510 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-61185ebb-b275-486e-8fa6-b757b1054ab8 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:14.267835 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-61185ebb-b275-486e-8fa6-b757b1054ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:14.288442 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:14.297226 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d147bb-9b4a-415a-b38f-da5422265f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:14.297925 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d147bb-9b4a-415a-b38f-da5422265f45 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:14.298559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d147bb-9b4a-415a-b38f-da5422265f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:14.298559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d147bb-9b4a-415a-b38f-da5422265f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:14.300443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e6d147bb-9b4a-415a-b38f-da5422265f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:14.368915 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1289/2550] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:14.379801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61185ebb-b275-486e-8fa6-b757b1054ab8 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:14.381279 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-61185ebb-b275-486e-8fa6-b757b1054ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:14.392310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d147bb-9b4a-415a-b38f-da5422265f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:14.394715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61185ebb-b275-486e-8fa6-b757b1054ab8 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:14.395552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d147bb-9b4a-415a-b38f-da5422265f45 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:14.399355 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b56fbc-ebbb-4da5-8958-45fce5de6795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:14.403680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b56fbc-ebbb-4da5-8958-45fce5de6795 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:14.403680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b56fbc-ebbb-4da5-8958-45fce5de6795 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:14.403680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b56fbc-ebbb-4da5-8958-45fce5de6795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:14.411689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b56fbc-ebbb-4da5-8958-45fce5de6795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:14.413155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b56fbc-ebbb-4da5-8958-45fce5de6795 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:14.414002 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1262/2551] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:03:14.425334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1290/2552] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:15.086658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d147bb-9b4a-415a-b38f-da5422265f45 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:15.088427 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e6d147bb-9b4a-415a-b38f-da5422265f45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:15.106577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d147bb-9b4a-415a-b38f-da5422265f45 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:15.112020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:15.116714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:15.116714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:15.116714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] PATH_INFO: `/v3/projects/fd982842e23d4e95975242444172c788` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:15.116714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd982842e23d4e95975242444172c788)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:15.117780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:15.213969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd982842e23d4e95975242444172c788', 'request_id': 'req-bba4540d-b259-42d4-a00d-589602fd181c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:15.292679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd982842e23d4e95975242444172c788', 'request_id': 'req-bba4540d-b259-42d4-a00d-589602fd181c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:15.302153 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd982842e23d4e95975242444172c788', 'request_id': 'req-bba4540d-b259-42d4-a00d-589602fd181c'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:15.314492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] The token cache is being invalidate because project fd982842e23d4e95975242444172c788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:15.314788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:15.315454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-bba4540d-b259-42d4-a00d-589602fd181c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:15.316983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1263/2553] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:15 2022] DELETE /identity/v3/projects/fd982842e23d4e95975242444172c788 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:15.321937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5123fe4-6f15-4ec7-9f74-ea9dfb64f035 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:15.322455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5123fe4-6f15-4ec7-9f74-ea9dfb64f035 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:15.322573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5123fe4-6f15-4ec7-9f74-ea9dfb64f035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:15.342907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-c5123fe4-6f15-4ec7-9f74-ea9dfb64f035 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:15.344273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c5123fe4-6f15-4ec7-9f74-ea9dfb64f035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:15.410952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1291/2554] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:15.418825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc2b33-a558-4293-9527-937f63ebbf7f None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:15.419130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc2b33-a558-4293-9527-937f63ebbf7f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:15.419405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc2b33-a558-4293-9527-937f63ebbf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:15.438368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-5abc2b33-a558-4293-9527-937f63ebbf7f None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:15.439849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5abc2b33-a558-4293-9527-937f63ebbf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:15.449932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1264/2555] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:15.454139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5123fe4-6f15-4ec7-9f74-ea9dfb64f035 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:15.465553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5123fe4-6f15-4ec7-9f74-ea9dfb64f035 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:15.471838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f010417-e1d6-4573-883c-9edd42cc70a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:15.472625 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f010417-e1d6-4573-883c-9edd42cc70a2 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:15.472986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f010417-e1d6-4573-883c-9edd42cc70a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:15.473519 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f010417-e1d6-4573-883c-9edd42cc70a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:15.474249 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f010417-e1d6-4573-883c-9edd42cc70a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:03:15.474567 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f010417-e1d6-4573-883c-9edd42cc70a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:15.476263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f010417-e1d6-4573-883c-9edd42cc70a2 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:15.527743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1292/2556] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:03:15.528749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc2b33-a558-4293-9527-937f63ebbf7f None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:15.542274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc2b33-a558-4293-9527-937f63ebbf7f None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:15.548975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c270cb-fd39-47dc-ad0d-9252a77e8148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:15.550360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c270cb-fd39-47dc-ad0d-9252a77e8148 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:15.550360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c270cb-fd39-47dc-ad0d-9252a77e8148 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:15.550740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c270cb-fd39-47dc-ad0d-9252a77e8148 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:15.551534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c270cb-fd39-47dc-ad0d-9252a77e8148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:15.553681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c270cb-fd39-47dc-ad0d-9252a77e8148 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:15.658727 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f010417-e1d6-4573-883c-9edd42cc70a2 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:15.660614 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-1f010417-e1d6-4573-883c-9edd42cc70a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:15.663497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1265/2557] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:15 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:03:15.670747 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c270cb-fd39-47dc-ad0d-9252a77e8148 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:15.691681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:15.742201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c270cb-fd39-47dc-ad0d-9252a77e8148 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:15.747308 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51d53c-86da-4e7e-8b70-6cbfbfea0245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:15.748206 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51d53c-86da-4e7e-8b70-6cbfbfea0245 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:15.748898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51d53c-86da-4e7e-8b70-6cbfbfea0245 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:15.750283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51d53c-86da-4e7e-8b70-6cbfbfea0245 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:15.751139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51d53c-86da-4e7e-8b70-6cbfbfea0245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:15.753548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51d53c-86da-4e7e-8b70-6cbfbfea0245 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:15.767096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f010417-e1d6-4573-883c-9edd42cc70a2 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:15.805883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:15.840623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3ef8b-ec38-4088-88b5-eeda27f7ee9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:15.841499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3ef8b-ec38-4088-88b5-eeda27f7ee9d service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:15.842258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3ef8b-ec38-4088-88b5-eeda27f7ee9d service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:15.842258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3ef8b-ec38-4088-88b5-eeda27f7ee9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:15.844504 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3da3ef8b-ec38-4088-88b5-eeda27f7ee9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:15.852657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1266/2558] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:15 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:15.857007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51d53c-86da-4e7e-8b70-6cbfbfea0245 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:15.859588 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3ef8b-ec38-4088-88b5-eeda27f7ee9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:15.861574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3ef8b-ec38-4088-88b5-eeda27f7ee9d service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:15.867741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51d53c-86da-4e7e-8b70-6cbfbfea0245 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:15.874537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791ccc2-eaef-4ac4-a9b4-135b1b112f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:15.875273 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791ccc2-eaef-4ac4-a9b4-135b1b112f4a admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:15.875563 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791ccc2-eaef-4ac4-a9b4-135b1b112f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:15.875898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791ccc2-eaef-4ac4-a9b4-135b1b112f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:15.876623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791ccc2-eaef-4ac4-a9b4-135b1b112f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:15.878593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791ccc2-eaef-4ac4-a9b4-135b1b112f4a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:15.879320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1293/2559] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:03:15.890066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1267/2560] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:03:15.894498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3ef8b-ec38-4088-88b5-eeda27f7ee9d service nova] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:15.908906 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3ef8b-ec38-4088-88b5-eeda27f7ee9d service nova] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:15.914300 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ad70b-7db4-4c87-9697-ffda2dd84571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:15.915005 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ad70b-7db4-4c87-9697-ffda2dd84571 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:15.915217 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ad70b-7db4-4c87-9697-ffda2dd84571 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:15.915497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ad70b-7db4-4c87-9697-ffda2dd84571 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:15.916222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ad70b-7db4-4c87-9697-ffda2dd84571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:15.917927 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ad70b-7db4-4c87-9697-ffda2dd84571 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:15.940328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791ccc2-eaef-4ac4-a9b4-135b1b112f4a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:15.943558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1294/2561] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:15.976042 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791ccc2-eaef-4ac4-a9b4-135b1b112f4a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:15.992704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0f235-d1a7-4c14-9644-aa54ac6e3191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:15.993858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0f235-d1a7-4c14-9644-aa54ac6e3191 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:15.994224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0f235-d1a7-4c14-9644-aa54ac6e3191 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:15.994651 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0f235-d1a7-4c14-9644-aa54ac6e3191 admin admin] PATH_INFO: `/v3/projects/65d54f050d8f49519224d7612bb68a4d/users/ac9dcdac9fdf4b35be1db83f09f9a354/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:16.016882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0f235-d1a7-4c14-9644-aa54ac6e3191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d54f050d8f49519224d7612bb68a4d, user_id=ac9dcdac9fdf4b35be1db83f09f9a354, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:16.019886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0f235-d1a7-4c14-9644-aa54ac6e3191 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:16.107879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1268/2562] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:15 2022] PUT /identity/v3/projects/65d54f050d8f49519224d7612bb68a4d/users/ac9dcdac9fdf4b35be1db83f09f9a354/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:16.108525 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea90f1c-b07c-42b7-ae2b-de7c89335ff5 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:16.108824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea90f1c-b07c-42b7-ae2b-de7c89335ff5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:16.109104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea90f1c-b07c-42b7-ae2b-de7c89335ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:16.138549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-0ea90f1c-b07c-42b7-ae2b-de7c89335ff5 None None] MFA Rules not processed for user `ac9dcdac9fdf4b35be1db83f09f9a354`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:16.139787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-0ea90f1c-b07c-42b7-ae2b-de7c89335ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:16.216713 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1295/2563] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:16.221006 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0f235-d1a7-4c14-9644-aa54ac6e3191 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:16.286464 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0f235-d1a7-4c14-9644-aa54ac6e3191 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:16.292405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce890b-fc22-4c1d-b0fe-aac253acdb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:16.293094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce890b-fc22-4c1d-b0fe-aac253acdb07 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:16.293451 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce890b-fc22-4c1d-b0fe-aac253acdb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:16.293732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce890b-fc22-4c1d-b0fe-aac253acdb07 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:16.294433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce890b-fc22-4c1d-b0fe-aac253acdb07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:16.296379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ce890b-fc22-4c1d-b0fe-aac253acdb07 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:16.404400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1269/2564] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:16 2022] POST /identity/v3/projects => generated 371 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:16.410452 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea90f1c-b07c-42b7-ae2b-de7c89335ff5 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:16.422791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea90f1c-b07c-42b7-ae2b-de7c89335ff5 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:16.488287 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a4369-f857-4054-adc6-f659f1779ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:16.489226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a4369-f857-4054-adc6-f659f1779ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:16.489653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a4369-f857-4054-adc6-f659f1779ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:16.489938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a4369-f857-4054-adc6-f659f1779ab4 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:16.490650 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a4369-f857-4054-adc6-f659f1779ab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:16.492633 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a4369-f857-4054-adc6-f659f1779ab4 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:16.581579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1296/2565] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:16 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:03:16.582921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce890b-fc22-4c1d-b0fe-aac253acdb07 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:16.594043 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce890b-fc22-4c1d-b0fe-aac253acdb07 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:16.598620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fe6c1-8ce4-4888-80ee-1f8c2a40dbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:16.599291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fe6c1-8ce4-4888-80ee-1f8c2a40dbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:16.599581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fe6c1-8ce4-4888-80ee-1f8c2a40dbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:16.599850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fe6c1-8ce4-4888-80ee-1f8c2a40dbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:16.600703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fe6c1-8ce4-4888-80ee-1f8c2a40dbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:16.602356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23fe6c1-8ce4-4888-80ee-1f8c2a40dbd0 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:16.618131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1270/2566] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:16.619263 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a4369-f857-4054-adc6-f659f1779ab4 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:16.631231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a4369-f857-4054-adc6-f659f1779ab4 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:16.636729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bf7a1-f2b1-49f2-820a-de47e322be8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:16.637582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bf7a1-f2b1-49f2-820a-de47e322be8a admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:16.638049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bf7a1-f2b1-49f2-820a-de47e322be8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:16.638352 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97bf7a1-f2b1-49f2-820a-de47e322be8a admin admin] PATH_INFO: `/v3/projects/e6a8ec237f114d0b9427f3a9eda4583e/users/12479fbf6f5344b7a163bc8dd5fac5fa/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:16.656503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bf7a1-f2b1-49f2-820a-de47e322be8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a8ec237f114d0b9427f3a9eda4583e, user_id=12479fbf6f5344b7a163bc8dd5fac5fa, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:16.659252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97bf7a1-f2b1-49f2-820a-de47e322be8a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:16.756244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1297/2567] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:16 2022] PUT /identity/v3/projects/e6a8ec237f114d0b9427f3a9eda4583e/users/12479fbf6f5344b7a163bc8dd5fac5fa/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:03:16.760366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fe6c1-8ce4-4888-80ee-1f8c2a40dbd0 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:16.771096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23fe6c1-8ce4-4888-80ee-1f8c2a40dbd0 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:16.826772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:16.840283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81a836-af2c-4484-b5cc-af04a6ccec92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:16.840954 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a836-af2c-4484-b5cc-af04a6ccec92 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:16.841261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a836-af2c-4484-b5cc-af04a6ccec92 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:16.841543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81a836-af2c-4484-b5cc-af04a6ccec92 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:16.842256 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81a836-af2c-4484-b5cc-af04a6ccec92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:16.843846 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81a836-af2c-4484-b5cc-af04a6ccec92 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:16.854642 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1271/2568] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:16.858615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bf7a1-f2b1-49f2-820a-de47e322be8a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:16.865397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:16.869325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97bf7a1-f2b1-49f2-820a-de47e322be8a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:16.874162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70c713-8756-49fd-990b-e363e94333bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:16.881336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70c713-8756-49fd-990b-e363e94333bb admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:16.881336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70c713-8756-49fd-990b-e363e94333bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:16.881336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70c713-8756-49fd-990b-e363e94333bb admin admin] PATH_INFO: `/v3/projects/e6a8ec237f114d0b9427f3a9eda4583e/users/12479fbf6f5344b7a163bc8dd5fac5fa/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:16.881336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70c713-8756-49fd-990b-e363e94333bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a8ec237f114d0b9427f3a9eda4583e, user_id=12479fbf6f5344b7a163bc8dd5fac5fa, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:16.881336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70c713-8756-49fd-990b-e363e94333bb admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:16.891866 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1298/2569] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:16 2022] PUT /identity/v3/projects/e6a8ec237f114d0b9427f3a9eda4583e/users/12479fbf6f5344b7a163bc8dd5fac5fa/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:16.904881 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebe0c5-66d4-48b0-8e24-c88487aed3a3 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:16.905173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebe0c5-66d4-48b0-8e24-c88487aed3a3 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:16.905557 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebe0c5-66d4-48b0-8e24-c88487aed3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:16.934344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-61ebe0c5-66d4-48b0-8e24-c88487aed3a3 None None] MFA Rules not processed for user `12479fbf6f5344b7a163bc8dd5fac5fa`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:16.935573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-61ebe0c5-66d4-48b0-8e24-c88487aed3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:17.005731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1272/2570] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:16 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:17.018002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70c713-8756-49fd-990b-e363e94333bb admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:17.019721 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6c70c713-8756-49fd-990b-e363e94333bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:17.080523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70c713-8756-49fd-990b-e363e94333bb admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:17.084294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2849f6-b28a-479a-b389-ff24c451940b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:17.084952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2849f6-b28a-479a-b389-ff24c451940b service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:17.085233 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2849f6-b28a-479a-b389-ff24c451940b service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:17.085553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2849f6-b28a-479a-b389-ff24c451940b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:17.095329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2849f6-b28a-479a-b389-ff24c451940b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:17.096803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2849f6-b28a-479a-b389-ff24c451940b service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:17.107406 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1299/2571] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:18.030855 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebe0c5-66d4-48b0-8e24-c88487aed3a3 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:18.100893 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebe0c5-66d4-48b0-8e24-c88487aed3a3 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:18.105546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:18.106429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:18.106717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:18.106997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] PATH_INFO: `/v3/users/ac9dcdac9fdf4b35be1db83f09f9a354` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:18.109411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac9dcdac9fdf4b35be1db83f09f9a354)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:18.111578 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:18.372396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac9dcdac9fdf4b35be1db83f09f9a354'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:18.380143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac9dcdac9fdf4b35be1db83f09f9a354'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:18.380874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac9dcdac9fdf4b35be1db83f09f9a354'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:18.393326 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:18.398400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac9dcdac9fdf4b35be1db83f09f9a354'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:18.401272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac9dcdac9fdf4b35be1db83f09f9a354'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:18.481498 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1273/2572] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:18 2022] DELETE /identity/v3/users/ac9dcdac9fdf4b35be1db83f09f9a354 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:18.486139 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be557b-f64c-4951-9388-b57711cbc68a None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:18.486624 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be557b-f64c-4951-9388-b57711cbc68a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:18.486733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be557b-f64c-4951-9388-b57711cbc68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:18.507134 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-33be557b-f64c-4951-9388-b57711cbc68a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:18.508375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-33be557b-f64c-4951-9388-b57711cbc68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:18.570895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1300/2573] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:18.581674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:18.585463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:18.645154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c5e0f-8fc5-4247-a55a-1d833b2de850 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:18.649585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03857ed3-7b9b-4ae1-9478-dd473085db16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:18.650236 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03857ed3-7b9b-4ae1-9478-dd473085db16 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:18.650522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03857ed3-7b9b-4ae1-9478-dd473085db16 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:18.650798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03857ed3-7b9b-4ae1-9478-dd473085db16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:18.660313 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03857ed3-7b9b-4ae1-9478-dd473085db16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:18.661832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03857ed3-7b9b-4ae1-9478-dd473085db16 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:18.671842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1274/2574] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:03:19.076892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be557b-f64c-4951-9388-b57711cbc68a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:19.078376 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-33be557b-f64c-4951-9388-b57711cbc68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:19.091150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be557b-f64c-4951-9388-b57711cbc68a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:19.098874 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:19.099761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:19.100049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:19.100349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] PATH_INFO: `/v3/projects/65d54f050d8f49519224d7612bb68a4d` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:19.101545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d54f050d8f49519224d7612bb68a4d)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:19.103549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:19.219933 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d54f050d8f49519224d7612bb68a4d', 'request_id': 'req-d80f5a69-4649-47af-9868-9b2953d107d6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:19.286836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d54f050d8f49519224d7612bb68a4d', 'request_id': 'req-d80f5a69-4649-47af-9868-9b2953d107d6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:19.293562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d54f050d8f49519224d7612bb68a4d', 'request_id': 'req-d80f5a69-4649-47af-9868-9b2953d107d6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:19.305115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] The token cache is being invalidate because project 65d54f050d8f49519224d7612bb68a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:19.305446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:19.306141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:19.307729 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1301/2575] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:19 2022] DELETE /identity/v3/projects/65d54f050d8f49519224d7612bb68a4d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:19.312015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03857ed3-7b9b-4ae1-9478-dd473085db16 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:19.322040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03857ed3-7b9b-4ae1-9478-dd473085db16 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:19.379390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677274f4-b896-4d0c-bc97-18f8e474c87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:19.380285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677274f4-b896-4d0c-bc97-18f8e474c87a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:19.380583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677274f4-b896-4d0c-bc97-18f8e474c87a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:19.380844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677274f4-b896-4d0c-bc97-18f8e474c87a admin admin] PATH_INFO: `/v3/users/12479fbf6f5344b7a163bc8dd5fac5fa` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:19.382133 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677274f4-b896-4d0c-bc97-18f8e474c87a admin admin] RBAC: Authorizing `identity:delete_user(user_id=12479fbf6f5344b7a163bc8dd5fac5fa)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:19.384110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677274f4-b896-4d0c-bc97-18f8e474c87a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:19.643959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-677274f4-b896-4d0c-bc97-18f8e474c87a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12479fbf6f5344b7a163bc8dd5fac5fa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:19.650742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-677274f4-b896-4d0c-bc97-18f8e474c87a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12479fbf6f5344b7a163bc8dd5fac5fa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:19.652021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-677274f4-b896-4d0c-bc97-18f8e474c87a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12479fbf6f5344b7a163bc8dd5fac5fa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:19.665176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-677274f4-b896-4d0c-bc97-18f8e474c87a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12479fbf6f5344b7a163bc8dd5fac5fa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:19.665972 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-677274f4-b896-4d0c-bc97-18f8e474c87a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12479fbf6f5344b7a163bc8dd5fac5fa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:19.756345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1275/2576] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:19 2022] DELETE /identity/v3/users/12479fbf6f5344b7a163bc8dd5fac5fa => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:20.174348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:20.176056 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:20.193605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f5a69-4649-47af-9868-9b2953d107d6 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:20.204432 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:20.250318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:20.251171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:20.251455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:20.251730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] PATH_INFO: `/v3/projects/e6a8ec237f114d0b9427f3a9eda4583e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:20.253194 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a8ec237f114d0b9427f3a9eda4583e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:20.255141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:20.368010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a8ec237f114d0b9427f3a9eda4583e', 'request_id': 'req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:20.452305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a8ec237f114d0b9427f3a9eda4583e', 'request_id': 'req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:20.458347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a8ec237f114d0b9427f3a9eda4583e', 'request_id': 'req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:20.469895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] The token cache is being invalidate because project e6a8ec237f114d0b9427f3a9eda4583e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:20.470184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:20.470865 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c89e93c3-eb7f-43ab-b4eb-8b19943a702a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:20.472392 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1302/2577] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:20 2022] DELETE /identity/v3/projects/e6a8ec237f114d0b9427f3a9eda4583e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:20.477583 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed0a0d-d77f-42a5-bf6c-6e5ed563ac45 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:20.478063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed0a0d-d77f-42a5-bf6c-6e5ed563ac45 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:20.478281 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed0a0d-d77f-42a5-bf6c-6e5ed563ac45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:20.491063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:20.497714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2bed0a0d-d77f-42a5-bf6c-6e5ed563ac45 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:20.498930 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2bed0a0d-d77f-42a5-bf6c-6e5ed563ac45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:20.558969 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1276/2578] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:20.567339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec976d16-eb2e-4973-ac2e-d9d8e9d541db None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:20.567631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec976d16-eb2e-4973-ac2e-d9d8e9d541db None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:20.568031 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec976d16-eb2e-4973-ac2e-d9d8e9d541db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:20.587974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-ec976d16-eb2e-4973-ac2e-d9d8e9d541db None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:20.589786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ec976d16-eb2e-4973-ac2e-d9d8e9d541db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:20.597862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1303/2579] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:20.603595 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed0a0d-d77f-42a5-bf6c-6e5ed563ac45 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:20.618175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed0a0d-d77f-42a5-bf6c-6e5ed563ac45 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:20.622971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdab4f5-d995-4e95-b30c-c92d7ec15fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:20.623689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab4f5-d995-4e95-b30c-c92d7ec15fcf admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:20.623975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab4f5-d995-4e95-b30c-c92d7ec15fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:20.624254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab4f5-d995-4e95-b30c-c92d7ec15fcf admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:20.625203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdab4f5-d995-4e95-b30c-c92d7ec15fcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:03:20.625641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdab4f5-d995-4e95-b30c-c92d7ec15fcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:20.627248 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdab4f5-d995-4e95-b30c-c92d7ec15fcf admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:20.640649 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1277/2580] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:03:20.644576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec976d16-eb2e-4973-ac2e-d9d8e9d541db None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:20.654908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec976d16-eb2e-4973-ac2e-d9d8e9d541db None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:20.660208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78715c7b-1a2e-4ca8-ae8c-d5b03d856b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:20.660952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78715c7b-1a2e-4ca8-ae8c-d5b03d856b58 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:20.661309 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78715c7b-1a2e-4ca8-ae8c-d5b03d856b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:20.661511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78715c7b-1a2e-4ca8-ae8c-d5b03d856b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:20.662199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78715c7b-1a2e-4ca8-ae8c-d5b03d856b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:20.664124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78715c7b-1a2e-4ca8-ae8c-d5b03d856b58 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:20.763308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1304/2581] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:20 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:20.767903 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdab4f5-d995-4e95-b30c-c92d7ec15fcf admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:20.779079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdab4f5-d995-4e95-b30c-c92d7ec15fcf admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:20.837316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10f594-267b-4ebb-8c5b-1f4ecdc4093f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:20.838044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10f594-267b-4ebb-8c5b-1f4ecdc4093f admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:20.838422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10f594-267b-4ebb-8c5b-1f4ecdc4093f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:20.839234 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10f594-267b-4ebb-8c5b-1f4ecdc4093f admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:20.845981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10f594-267b-4ebb-8c5b-1f4ecdc4093f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:20.848021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10f594-267b-4ebb-8c5b-1f4ecdc4093f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:20.938908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1278/2582] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:20 2022] POST /identity/v3/users => generated 283 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:20.942971 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78715c7b-1a2e-4ca8-ae8c-d5b03d856b58 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:20.953820 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78715c7b-1a2e-4ca8-ae8c-d5b03d856b58 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:20.958502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed7c29-7828-4895-94d9-38647ca2bf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:20.959169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed7c29-7828-4895-94d9-38647ca2bf6d admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:20.959548 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed7c29-7828-4895-94d9-38647ca2bf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:20.959744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed7c29-7828-4895-94d9-38647ca2bf6d admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:20.960503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed7c29-7828-4895-94d9-38647ca2bf6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:20.962176 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed7c29-7828-4895-94d9-38647ca2bf6d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:20.987102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1305/2583] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:20.991627 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10f594-267b-4ebb-8c5b-1f4ecdc4093f admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:20.992332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed7c29-7828-4895-94d9-38647ca2bf6d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:20.994220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3bed7c29-7828-4895-94d9-38647ca2bf6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:21.002309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10f594-267b-4ebb-8c5b-1f4ecdc4093f admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:21.007750 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209bb53-74f8-480b-9873-a1f27c861b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:21.008492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209bb53-74f8-480b-9873-a1f27c861b10 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:21.008776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209bb53-74f8-480b-9873-a1f27c861b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:21.009052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209bb53-74f8-480b-9873-a1f27c861b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:21.010050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209bb53-74f8-480b-9873-a1f27c861b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:21.012208 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209bb53-74f8-480b-9873-a1f27c861b10 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:21.023088 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1279/2584] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:21.027719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209bb53-74f8-480b-9873-a1f27c861b10 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:21.038299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209bb53-74f8-480b-9873-a1f27c861b10 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:21.043178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c26b6e-f199-45a8-b49c-49cc3246260b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:21.044090 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c26b6e-f199-45a8-b49c-49cc3246260b admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:21.044435 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c26b6e-f199-45a8-b49c-49cc3246260b admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:21.044712 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c26b6e-f199-45a8-b49c-49cc3246260b admin admin] PATH_INFO: `/v3/projects/87a5a5113eee4c67807bef423a58038f/users/89c0d19ab50945ca97275d8a6176cfb4/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:21.055419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed7c29-7828-4895-94d9-38647ca2bf6d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:21.059511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4b458-7805-40b7-aaa5-d0b1d1bfeb81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:21.060163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4b458-7805-40b7-aaa5-d0b1d1bfeb81 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:21.060460 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4b458-7805-40b7-aaa5-d0b1d1bfeb81 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:21.060733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4b458-7805-40b7-aaa5-d0b1d1bfeb81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:21.062507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c26b6e-f199-45a8-b49c-49cc3246260b admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a5a5113eee4c67807bef423a58038f, user_id=89c0d19ab50945ca97275d8a6176cfb4, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:21.062647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e1b4b458-7805-40b7-aaa5-d0b1d1bfeb81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:21.065166 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c26b6e-f199-45a8-b49c-49cc3246260b admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:21.076480 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4b458-7805-40b7-aaa5-d0b1d1bfeb81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:21.077970 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4b458-7805-40b7-aaa5-d0b1d1bfeb81 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:21.088239 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1306/2585] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:03:21.134170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1280/2586] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:21 2022] PUT /identity/v3/projects/87a5a5113eee4c67807bef423a58038f/users/89c0d19ab50945ca97275d8a6176cfb4/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:03:21.146697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea51ec-4942-4ceb-ac8b-09083e7197cf None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:21.147096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea51ec-4942-4ceb-ac8b-09083e7197cf None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:21.147389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea51ec-4942-4ceb-ac8b-09083e7197cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:21.168468 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:21.176633 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-bbea51ec-4942-4ceb-ac8b-09083e7197cf None None] MFA Rules not processed for user `89c0d19ab50945ca97275d8a6176cfb4`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:21.177920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-bbea51ec-4942-4ceb-ac8b-09083e7197cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:21.245081 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1307/2587] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:21.249224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c26b6e-f199-45a8-b49c-49cc3246260b admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:21.306016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:21.319819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c26b6e-f199-45a8-b49c-49cc3246260b admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:21.325466 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e98b8-8bfe-4484-b084-34aa5f30c5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:21.326001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e98b8-8bfe-4484-b084-34aa5f30c5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:21.327079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e98b8-8bfe-4484-b084-34aa5f30c5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:21.327604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e98b8-8bfe-4484-b084-34aa5f30c5b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:21.328769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e98b8-8bfe-4484-b084-34aa5f30c5b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:21.330832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e98b8-8bfe-4484-b084-34aa5f30c5b1 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:21.429999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1281/2588] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:21 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:21.431449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea51ec-4942-4ceb-ac8b-09083e7197cf None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:21.443220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea51ec-4942-4ceb-ac8b-09083e7197cf None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:21.506926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e978000-c66e-4805-898e-44662b1ea069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:21.507612 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e978000-c66e-4805-898e-44662b1ea069 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:21.507900 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e978000-c66e-4805-898e-44662b1ea069 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:21.508185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e978000-c66e-4805-898e-44662b1ea069 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:21.508967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e978000-c66e-4805-898e-44662b1ea069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:21.511700 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e978000-c66e-4805-898e-44662b1ea069 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:21.598748 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1308/2589] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:21 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:21.599905 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e98b8-8bfe-4484-b084-34aa5f30c5b1 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:21.610219 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e98b8-8bfe-4484-b084-34aa5f30c5b1 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:21.614945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c96d1d-98ee-4ec7-b293-03ce4c3badee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:21.615603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c96d1d-98ee-4ec7-b293-03ce4c3badee admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:21.615883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c96d1d-98ee-4ec7-b293-03ce4c3badee admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:21.616157 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c96d1d-98ee-4ec7-b293-03ce4c3badee admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:21.616938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c96d1d-98ee-4ec7-b293-03ce4c3badee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:21.618580 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c96d1d-98ee-4ec7-b293-03ce4c3badee admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:21.631774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1282/2590] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:03:21.637681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e978000-c66e-4805-898e-44662b1ea069 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:21.648047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e978000-c66e-4805-898e-44662b1ea069 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:21.653152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d2ff0-328e-4287-8282-376a32b274aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:21.654144 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d2ff0-328e-4287-8282-376a32b274aa admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:21.654528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d2ff0-328e-4287-8282-376a32b274aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:21.654881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d2ff0-328e-4287-8282-376a32b274aa admin admin] PATH_INFO: `/v3/projects/c27e00b377114945a80d9708b174ea5a/users/30421be7801b401b87b2a8ee121e2cd9/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:21.671705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d2ff0-328e-4287-8282-376a32b274aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c27e00b377114945a80d9708b174ea5a, user_id=30421be7801b401b87b2a8ee121e2cd9, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:21.674402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d2ff0-328e-4287-8282-376a32b274aa admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:21.720947 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1309/2591] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:21 2022] PUT /identity/v3/projects/c27e00b377114945a80d9708b174ea5a/users/30421be7801b401b87b2a8ee121e2cd9/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:21.724981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c96d1d-98ee-4ec7-b293-03ce4c3badee admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:21.735820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c96d1d-98ee-4ec7-b293-03ce4c3badee admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:21.791483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba48763-963f-4b96-aec1-edd0e9cedaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:21.792664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba48763-963f-4b96-aec1-edd0e9cedaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:21.792947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba48763-963f-4b96-aec1-edd0e9cedaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:21.793221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba48763-963f-4b96-aec1-edd0e9cedaf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:21.793959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba48763-963f-4b96-aec1-edd0e9cedaf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:21.795626 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba48763-963f-4b96-aec1-edd0e9cedaf7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:21.805774 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1283/2592] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:21.810487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d2ff0-328e-4287-8282-376a32b274aa admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:21.820788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d2ff0-328e-4287-8282-376a32b274aa admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:21.825568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab233dd9-84e8-4826-88e4-70d775178340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:21.826459 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab233dd9-84e8-4826-88e4-70d775178340 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:21.826743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab233dd9-84e8-4826-88e4-70d775178340 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:21.827019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab233dd9-84e8-4826-88e4-70d775178340 admin admin] PATH_INFO: `/v3/projects/c27e00b377114945a80d9708b174ea5a/users/30421be7801b401b87b2a8ee121e2cd9/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:21.829097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab233dd9-84e8-4826-88e4-70d775178340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c27e00b377114945a80d9708b174ea5a, user_id=30421be7801b401b87b2a8ee121e2cd9, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:21.831757 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab233dd9-84e8-4826-88e4-70d775178340 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:21.842064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1310/2593] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:21 2022] PUT /identity/v3/projects/c27e00b377114945a80d9708b174ea5a/users/30421be7801b401b87b2a8ee121e2cd9/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:21.852217 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417cc21-0e9c-4ba3-b1e6-49adf4085014 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:21.852449 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417cc21-0e9c-4ba3-b1e6-49adf4085014 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:21.852740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417cc21-0e9c-4ba3-b1e6-49adf4085014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:21.880391 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-a417cc21-0e9c-4ba3-b1e6-49adf4085014 None None] MFA Rules not processed for user `30421be7801b401b87b2a8ee121e2cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:21.881718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a417cc21-0e9c-4ba3-b1e6-49adf4085014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:21.952408 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1284/2594] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:21.963837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab233dd9-84e8-4826-88e4-70d775178340 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:21.965557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ab233dd9-84e8-4826-88e4-70d775178340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:22.027098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab233dd9-84e8-4826-88e4-70d775178340 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:22.031019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a807f3-8974-4660-9fb0-988bc7af21d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:22.031675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a807f3-8974-4660-9fb0-988bc7af21d0 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:22.031964 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a807f3-8974-4660-9fb0-988bc7af21d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:22.032240 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a807f3-8974-4660-9fb0-988bc7af21d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:22.042438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:22.044826 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a807f3-8974-4660-9fb0-988bc7af21d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:22.046393 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a807f3-8974-4660-9fb0-988bc7af21d0 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:22.056518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1311/2595] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:22.933978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417cc21-0e9c-4ba3-b1e6-49adf4085014 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:22.990975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:23.009829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417cc21-0e9c-4ba3-b1e6-49adf4085014 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:23.015165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:23.016044 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:23.016397 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:23.016601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] PATH_INFO: `/v3/users/89c0d19ab50945ca97275d8a6176cfb4` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:23.017868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=89c0d19ab50945ca97275d8a6176cfb4)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:23.019916 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:23.214457 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89c0d19ab50945ca97275d8a6176cfb4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:23.221515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89c0d19ab50945ca97275d8a6176cfb4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:23.222231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89c0d19ab50945ca97275d8a6176cfb4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:23.236302 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89c0d19ab50945ca97275d8a6176cfb4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:23.237026 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89c0d19ab50945ca97275d8a6176cfb4'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:23.298963 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1285/2596] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:22 2022] DELETE /identity/v3/users/89c0d19ab50945ca97275d8a6176cfb4 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:23.303709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c5a86-9120-42b1-ace8-041133feb900 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:23.304166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c5a86-9120-42b1-ace8-041133feb900 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:23.304166 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c5a86-9120-42b1-ace8-041133feb900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:23.323887 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-7b4c5a86-9120-42b1-ace8-041133feb900 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:23.325097 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7b4c5a86-9120-42b1-ace8-041133feb900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:23.386078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1312/2597] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:23.395807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:23.397272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:23.455335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738bae7-ba04-45fe-a3b5-a062a616a4fd admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:23.459702 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596f0be-a24a-4cb9-aa42-fb3938817ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:23.460757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596f0be-a24a-4cb9-aa42-fb3938817ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:23.461288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596f0be-a24a-4cb9-aa42-fb3938817ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:23.461588 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596f0be-a24a-4cb9-aa42-fb3938817ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:23.471501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596f0be-a24a-4cb9-aa42-fb3938817ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:23.473125 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8596f0be-a24a-4cb9-aa42-fb3938817ac7 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:23.484448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1286/2598] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:24.066921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c5a86-9120-42b1-ace8-041133feb900 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:24.071128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7b4c5a86-9120-42b1-ace8-041133feb900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:24.099926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c5a86-9120-42b1-ace8-041133feb900 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:24.105188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:24.106192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:24.106549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:24.106898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] PATH_INFO: `/v3/projects/87a5a5113eee4c67807bef423a58038f` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:24.108333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a5a5113eee4c67807bef423a58038f)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:24.110317 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:24.216085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a5a5113eee4c67807bef423a58038f', 'request_id': 'req-c18d301c-180e-45ce-b8be-08e82834f382'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:24.284607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a5a5113eee4c67807bef423a58038f', 'request_id': 'req-c18d301c-180e-45ce-b8be-08e82834f382'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:24.289009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a5a5113eee4c67807bef423a58038f', 'request_id': 'req-c18d301c-180e-45ce-b8be-08e82834f382'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:24.306416 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] The token cache is being invalidate because project 87a5a5113eee4c67807bef423a58038f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:24.307053 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:24.308127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:24.313281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1313/2599] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:24 2022] DELETE /identity/v3/projects/87a5a5113eee4c67807bef423a58038f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:24.315484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596f0be-a24a-4cb9-aa42-fb3938817ac7 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:24.326028 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596f0be-a24a-4cb9-aa42-fb3938817ac7 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:24.426253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37941f73-5799-4f67-ad99-9a36db44dd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:24.427142 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37941f73-5799-4f67-ad99-9a36db44dd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:24.427428 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37941f73-5799-4f67-ad99-9a36db44dd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:24.427822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37941f73-5799-4f67-ad99-9a36db44dd68 admin admin] PATH_INFO: `/v3/users/30421be7801b401b87b2a8ee121e2cd9` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:24.429249 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37941f73-5799-4f67-ad99-9a36db44dd68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30421be7801b401b87b2a8ee121e2cd9)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:24.431207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37941f73-5799-4f67-ad99-9a36db44dd68 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:24.438040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:24.605140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-37941f73-5799-4f67-ad99-9a36db44dd68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30421be7801b401b87b2a8ee121e2cd9'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:24.619925 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-37941f73-5799-4f67-ad99-9a36db44dd68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30421be7801b401b87b2a8ee121e2cd9'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:24.620741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-37941f73-5799-4f67-ad99-9a36db44dd68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30421be7801b401b87b2a8ee121e2cd9'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:24.634834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-37941f73-5799-4f67-ad99-9a36db44dd68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30421be7801b401b87b2a8ee121e2cd9'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:24.635522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-37941f73-5799-4f67-ad99-9a36db44dd68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30421be7801b401b87b2a8ee121e2cd9'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:24.709975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1287/2600] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:24 2022] DELETE /identity/v3/users/30421be7801b401b87b2a8ee121e2cd9 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:25.214615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:25.215826 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:25.228658 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d301c-180e-45ce-b8be-08e82834f382 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:25.298202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:25.298978 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:25.299264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:25.299536 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] PATH_INFO: `/v3/projects/c27e00b377114945a80d9708b174ea5a` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:25.301261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c27e00b377114945a80d9708b174ea5a)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:25.303045 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:25.326125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:25.426402 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c27e00b377114945a80d9708b174ea5a', 'request_id': 'req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:25.487039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c27e00b377114945a80d9708b174ea5a', 'request_id': 'req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:25.493400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c27e00b377114945a80d9708b174ea5a', 'request_id': 'req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:25.506147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] The token cache is being invalidate because project c27e00b377114945a80d9708b174ea5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:25.506443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:25.507100 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-de59dae7-09cb-43bf-bdfa-c80266e9f1fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:25.508666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1314/2601] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:25 2022] DELETE /identity/v3/projects/c27e00b377114945a80d9708b174ea5a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:25.514349 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5de49-39fd-4a67-8fcd-6436697fa461 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:25.514881 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5de49-39fd-4a67-8fcd-6436697fa461 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:25.515110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5de49-39fd-4a67-8fcd-6436697fa461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:25.533883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-41b5de49-39fd-4a67-8fcd-6436697fa461 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:25.535107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-41b5de49-39fd-4a67-8fcd-6436697fa461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:25.595102 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1288/2602] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:25.603249 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efbe9b-695b-40c3-83ee-90871a33417a None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:25.603542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efbe9b-695b-40c3-83ee-90871a33417a None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:25.603816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efbe9b-695b-40c3-83ee-90871a33417a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:25.623275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-64efbe9b-695b-40c3-83ee-90871a33417a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:25.624512 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-64efbe9b-695b-40c3-83ee-90871a33417a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:25.632633 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1315/2603] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:25.636849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5de49-39fd-4a67-8fcd-6436697fa461 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:25.646606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5de49-39fd-4a67-8fcd-6436697fa461 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:25.651225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce9f6e-b701-4a09-9b16-a6fea276d122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:25.651883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce9f6e-b701-4a09-9b16-a6fea276d122 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:25.652173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce9f6e-b701-4a09-9b16-a6fea276d122 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:25.652453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce9f6e-b701-4a09-9b16-a6fea276d122 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:25.653176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce9f6e-b701-4a09-9b16-a6fea276d122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:03:25.653522 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce9f6e-b701-4a09-9b16-a6fea276d122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:25.655149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce9f6e-b701-4a09-9b16-a6fea276d122 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:25.666868 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1289/2604] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:03:25.671833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efbe9b-695b-40c3-83ee-90871a33417a None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:25.681561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efbe9b-695b-40c3-83ee-90871a33417a None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:25.686936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3354-7f93-4da6-8e99-60aa5c21a868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:25.687656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3354-7f93-4da6-8e99-60aa5c21a868 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:25.688032 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3354-7f93-4da6-8e99-60aa5c21a868 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:25.688368 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b3354-7f93-4da6-8e99-60aa5c21a868 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:25.689123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b3354-7f93-4da6-8e99-60aa5c21a868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:25.691163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b3354-7f93-4da6-8e99-60aa5c21a868 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:25.796981 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1316/2605] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:25 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:03:25.801048 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce9f6e-b701-4a09-9b16-a6fea276d122 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:25.811621 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce9f6e-b701-4a09-9b16-a6fea276d122 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:25.874799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6b63b-fbaa-4a19-8446-6fec6f439e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:25.875489 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6b63b-fbaa-4a19-8446-6fec6f439e09 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:25.875771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6b63b-fbaa-4a19-8446-6fec6f439e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:25.876138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6b63b-fbaa-4a19-8446-6fec6f439e09 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:25.876842 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6b63b-fbaa-4a19-8446-6fec6f439e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:25.879120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6b63b-fbaa-4a19-8446-6fec6f439e09 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:25.945661 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1290/2606] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:25 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:25.950157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3354-7f93-4da6-8e99-60aa5c21a868 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:25.961133 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b3354-7f93-4da6-8e99-60aa5c21a868 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:25.967150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51001d9b-b4f0-4e41-9389-9bdb34aca024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:25.967842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51001d9b-b4f0-4e41-9389-9bdb34aca024 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:25.968125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51001d9b-b4f0-4e41-9389-9bdb34aca024 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:25.968400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51001d9b-b4f0-4e41-9389-9bdb34aca024 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:25.969124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51001d9b-b4f0-4e41-9389-9bdb34aca024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:25.970775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51001d9b-b4f0-4e41-9389-9bdb34aca024 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:25.984885 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1317/2607] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:03:25.988798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6b63b-fbaa-4a19-8446-6fec6f439e09 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:25.994867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:26.001389 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6b63b-fbaa-4a19-8446-6fec6f439e09 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:26.010210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506181b-f89c-4fb7-8ede-9f80fcb9ffa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:26.012110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506181b-f89c-4fb7-8ede-9f80fcb9ffa7 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:26.012393 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506181b-f89c-4fb7-8ede-9f80fcb9ffa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:26.012669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506181b-f89c-4fb7-8ede-9f80fcb9ffa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:26.015696 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506181b-f89c-4fb7-8ede-9f80fcb9ffa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:26.016504 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506181b-f89c-4fb7-8ede-9f80fcb9ffa7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:26.033310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1291/2608] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:03:26.038155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51001d9b-b4f0-4e41-9389-9bdb34aca024 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:26.051904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51001d9b-b4f0-4e41-9389-9bdb34aca024 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:26.058744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dddf5fd-abae-444b-a030-65c0bc2de8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:26.060381 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddf5fd-abae-444b-a030-65c0bc2de8c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:26.060744 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddf5fd-abae-444b-a030-65c0bc2de8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:26.061103 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dddf5fd-abae-444b-a030-65c0bc2de8c5 admin admin] PATH_INFO: `/v3/projects/99eeaf8e613e4a4c908e3ba52ee900e8/users/7f49b5aa62e1466c870d1a2028496a9b/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:26.085091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dddf5fd-abae-444b-a030-65c0bc2de8c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99eeaf8e613e4a4c908e3ba52ee900e8, user_id=7f49b5aa62e1466c870d1a2028496a9b, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:26.088944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dddf5fd-abae-444b-a030-65c0bc2de8c5 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:26.196801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1318/2609] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:26 2022] PUT /identity/v3/projects/99eeaf8e613e4a4c908e3ba52ee900e8/users/7f49b5aa62e1466c870d1a2028496a9b/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:26.209224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e19179-b871-4042-9cb7-8478544f4755 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:26.209617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e19179-b871-4042-9cb7-8478544f4755 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:26.209890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e19179-b871-4042-9cb7-8478544f4755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:26.260086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b3e19179-b871-4042-9cb7-8478544f4755 None None] MFA Rules not processed for user `7f49b5aa62e1466c870d1a2028496a9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:26.261576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b3e19179-b871-4042-9cb7-8478544f4755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:26.341596 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1292/2610] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:26.346226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dddf5fd-abae-444b-a030-65c0bc2de8c5 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:26.426387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dddf5fd-abae-444b-a030-65c0bc2de8c5 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:26.431051 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c696cc4-0d42-429b-abba-e5f8658dde5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:26.431711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c696cc4-0d42-429b-abba-e5f8658dde5b admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:26.431996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c696cc4-0d42-429b-abba-e5f8658dde5b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:26.432270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c696cc4-0d42-429b-abba-e5f8658dde5b admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:26.433600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c696cc4-0d42-429b-abba-e5f8658dde5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:26.435556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c696cc4-0d42-429b-abba-e5f8658dde5b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:26.577351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1319/2611] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:26 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:26.581177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e19179-b871-4042-9cb7-8478544f4755 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:26.596964 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e19179-b871-4042-9cb7-8478544f4755 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:26.669097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75242671-d4c3-4820-b70a-9c201badb700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:26.670111 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75242671-d4c3-4820-b70a-9c201badb700 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:26.670638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75242671-d4c3-4820-b70a-9c201badb700 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:26.670948 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75242671-d4c3-4820-b70a-9c201badb700 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:26.671643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75242671-d4c3-4820-b70a-9c201badb700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:26.673654 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75242671-d4c3-4820-b70a-9c201badb700 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:26.749283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1293/2612] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:26 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:03:26.750871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c696cc4-0d42-429b-abba-e5f8658dde5b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:26.762365 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c696cc4-0d42-429b-abba-e5f8658dde5b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:26.767386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48af8a1-6726-473f-8d0c-5d7b31710877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:26.768086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48af8a1-6726-473f-8d0c-5d7b31710877 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:26.768379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48af8a1-6726-473f-8d0c-5d7b31710877 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:26.768665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48af8a1-6726-473f-8d0c-5d7b31710877 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:26.769931 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48af8a1-6726-473f-8d0c-5d7b31710877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:26.771076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48af8a1-6726-473f-8d0c-5d7b31710877 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:26.785538 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1320/2613] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:26.786794 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75242671-d4c3-4820-b70a-9c201badb700 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:26.798010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75242671-d4c3-4820-b70a-9c201badb700 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:26.803514 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe3e92-8943-4b17-bd19-7dda529a47cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:26.804468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe3e92-8943-4b17-bd19-7dda529a47cc admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:26.804867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe3e92-8943-4b17-bd19-7dda529a47cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:26.805158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe3e92-8943-4b17-bd19-7dda529a47cc admin admin] PATH_INFO: `/v3/projects/a315bb61d9a1403fa5d12c5273d7f289/users/2de97c3d7da04746af777035b7331c5c/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:26.827953 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe3e92-8943-4b17-bd19-7dda529a47cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a315bb61d9a1403fa5d12c5273d7f289, user_id=2de97c3d7da04746af777035b7331c5c, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:26.830756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe3e92-8943-4b17-bd19-7dda529a47cc admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:26.899549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1294/2614] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:26 2022] PUT /identity/v3/projects/a315bb61d9a1403fa5d12c5273d7f289/users/2de97c3d7da04746af777035b7331c5c/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:26.903606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48af8a1-6726-473f-8d0c-5d7b31710877 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:26.915490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48af8a1-6726-473f-8d0c-5d7b31710877 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:26.935768 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:26.983245 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905320e-46b2-4488-b884-f550b4c44e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:26.983861 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905320e-46b2-4488-b884-f550b4c44e3a admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:26.984147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905320e-46b2-4488-b884-f550b4c44e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:26.986405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905320e-46b2-4488-b884-f550b4c44e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:26.986405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905320e-46b2-4488-b884-f550b4c44e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:26.986997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905320e-46b2-4488-b884-f550b4c44e3a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:26.997871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1321/2615] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:03:27.002026 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe3e92-8943-4b17-bd19-7dda529a47cc admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:27.013501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe3e92-8943-4b17-bd19-7dda529a47cc admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:27.018120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af3ff0-088c-4f6c-987a-95cf5e61134d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:27.019003 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af3ff0-088c-4f6c-987a-95cf5e61134d admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:27.019290 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af3ff0-088c-4f6c-987a-95cf5e61134d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:27.019606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2af3ff0-088c-4f6c-987a-95cf5e61134d admin admin] PATH_INFO: `/v3/projects/a315bb61d9a1403fa5d12c5273d7f289/users/2de97c3d7da04746af777035b7331c5c/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:27.021882 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af3ff0-088c-4f6c-987a-95cf5e61134d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a315bb61d9a1403fa5d12c5273d7f289, user_id=2de97c3d7da04746af777035b7331c5c, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:27.024845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2af3ff0-088c-4f6c-987a-95cf5e61134d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:27.035710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1295/2616] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:27 2022] PUT /identity/v3/projects/a315bb61d9a1403fa5d12c5273d7f289/users/2de97c3d7da04746af777035b7331c5c/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:27.046549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00738e6-34be-474e-949b-3ab507c5fc12 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:27.047128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00738e6-34be-474e-949b-3ab507c5fc12 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:27.047573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00738e6-34be-474e-949b-3ab507c5fc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:27.073960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-e00738e6-34be-474e-949b-3ab507c5fc12 None None] MFA Rules not processed for user `2de97c3d7da04746af777035b7331c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:27.075224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e00738e6-34be-474e-949b-3ab507c5fc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:27.156754 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1322/2617] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:27.174073 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af3ff0-088c-4f6c-987a-95cf5e61134d admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:27.180149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f2af3ff0-088c-4f6c-987a-95cf5e61134d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:27.220246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:27.246798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2af3ff0-088c-4f6c-987a-95cf5e61134d admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:27.253706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44a49a-2aeb-4f96-a593-cbeedb91fd44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:27.254743 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44a49a-2aeb-4f96-a593-cbeedb91fd44 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:27.255366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44a49a-2aeb-4f96-a593-cbeedb91fd44 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:27.255822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44a49a-2aeb-4f96-a593-cbeedb91fd44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:27.326317 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44a49a-2aeb-4f96-a593-cbeedb91fd44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:27.327787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44a49a-2aeb-4f96-a593-cbeedb91fd44 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:27.337949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1296/2618] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:28.591879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00738e6-34be-474e-949b-3ab507c5fc12 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:28.593474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e00738e6-34be-474e-949b-3ab507c5fc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:28.664718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00738e6-34be-474e-949b-3ab507c5fc12 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:28.669829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:28.670699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:28.670984 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:28.671260 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] PATH_INFO: `/v3/users/0569765b6f07473fa07a5376b3677da8` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:28.672443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0569765b6f07473fa07a5376b3677da8)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:28.674461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:28.836225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:28.838814 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0569765b6f07473fa07a5376b3677da8'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:28.839479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0569765b6f07473fa07a5376b3677da8'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:28.846550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0569765b6f07473fa07a5376b3677da8'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:28.847226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0569765b6f07473fa07a5376b3677da8'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:28.863187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0569765b6f07473fa07a5376b3677da8'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:28.927305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1323/2619] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:28 2022] DELETE /identity/v3/users/0569765b6f07473fa07a5376b3677da8 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:29.230789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44a49a-2aeb-4f96-a593-cbeedb91fd44 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:29.244052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44a49a-2aeb-4f96-a593-cbeedb91fd44 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:29.341959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:29.342757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:29.343039 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:29.343315 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] PATH_INFO: `/v3/projects/5db8d64fab0e45ffbba87b1ae3b48703` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:29.344506 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db8d64fab0e45ffbba87b1ae3b48703)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:29.346452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:29.482643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db8d64fab0e45ffbba87b1ae3b48703', 'request_id': 'req-caeede46-241f-4f1e-9a88-0bb23840c930'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:29.595889 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db8d64fab0e45ffbba87b1ae3b48703', 'request_id': 'req-caeede46-241f-4f1e-9a88-0bb23840c930'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:29.602767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db8d64fab0e45ffbba87b1ae3b48703', 'request_id': 'req-caeede46-241f-4f1e-9a88-0bb23840c930'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:29.614666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] The token cache is being invalidate because project 5db8d64fab0e45ffbba87b1ae3b48703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:29.614956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:29.615643 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:29.617178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1297/2620] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:29 2022] DELETE /identity/v3/projects/5db8d64fab0e45ffbba87b1ae3b48703 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:29.621226 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:29.624445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:29.641180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bf95e-9acc-4fa5-b111-034a8a9ad576 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:29.699901 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:29.701149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:29.703390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:29.703390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] PATH_INFO: `/v3/users/68fab9785f514c5e9e45a0ec95d97604` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:29.703390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68fab9785f514c5e9e45a0ec95d97604)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:29.705734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:29.890745 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68fab9785f514c5e9e45a0ec95d97604'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:29.894611 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68fab9785f514c5e9e45a0ec95d97604'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:29.914493 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68fab9785f514c5e9e45a0ec95d97604'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:29.915514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68fab9785f514c5e9e45a0ec95d97604'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:29.931049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68fab9785f514c5e9e45a0ec95d97604'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:30.042065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1324/2621] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:29 2022] DELETE /identity/v3/users/68fab9785f514c5e9e45a0ec95d97604 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:30.312346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:30.314030 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:30.329653 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeede46-241f-4f1e-9a88-0bb23840c930 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:30.343414 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:30.388825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:30.389684 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:30.389973 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:30.390254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] PATH_INFO: `/v3/projects/3bae3e36253c4e47b7b507f53663186b` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:30.391434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bae3e36253c4e47b7b507f53663186b)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:30.393414 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:30.492933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bae3e36253c4e47b7b507f53663186b', 'request_id': 'req-429852e9-49a6-40f7-9ff3-eb23df387982'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:30.539224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bae3e36253c4e47b7b507f53663186b', 'request_id': 'req-429852e9-49a6-40f7-9ff3-eb23df387982'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:30.544853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bae3e36253c4e47b7b507f53663186b', 'request_id': 'req-429852e9-49a6-40f7-9ff3-eb23df387982'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:30.556052 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] The token cache is being invalidate because project 3bae3e36253c4e47b7b507f53663186b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:30.556339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:30.556963 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:30.559179 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1298/2622] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:30 2022] DELETE /identity/v3/projects/3bae3e36253c4e47b7b507f53663186b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:32.267537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:32.269076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:32.282407 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb85ea8-dcf1-4e47-90e8-237164085143 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:32.319201 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:32.337257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d32df-c12e-4935-9bcb-2d6f6cbb2efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:32.338188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d32df-c12e-4935-9bcb-2d6f6cbb2efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:32.338470 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d32df-c12e-4935-9bcb-2d6f6cbb2efa admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:32.338750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d32df-c12e-4935-9bcb-2d6f6cbb2efa admin admin] PATH_INFO: `/v3/users/b94704a420db42108fb35089605cbe22` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:32.340533 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d32df-c12e-4935-9bcb-2d6f6cbb2efa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b94704a420db42108fb35089605cbe22)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:32.342473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d32df-c12e-4935-9bcb-2d6f6cbb2efa admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:32.564379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b80d32df-c12e-4935-9bcb-2d6f6cbb2efa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b94704a420db42108fb35089605cbe22'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:32.565354 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b80d32df-c12e-4935-9bcb-2d6f6cbb2efa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b94704a420db42108fb35089605cbe22'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:32.572654 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b80d32df-c12e-4935-9bcb-2d6f6cbb2efa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b94704a420db42108fb35089605cbe22'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:32.573443 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b80d32df-c12e-4935-9bcb-2d6f6cbb2efa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b94704a420db42108fb35089605cbe22'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:32.586787 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-b80d32df-c12e-4935-9bcb-2d6f6cbb2efa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b94704a420db42108fb35089605cbe22'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:32.631283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:32.632680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:32.647275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1325/2623] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:32 2022] DELETE /identity/v3/users/b94704a420db42108fb35089605cbe22 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:32.651683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d0095-1f33-4409-8206-f4f4d4581e20 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:32.651985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d0095-1f33-4409-8206-f4f4d4581e20 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:32.652788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d0095-1f33-4409-8206-f4f4d4581e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:32.671492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-fd1d0095-1f33-4409-8206-f4f4d4581e20 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:32.672711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-fd1d0095-1f33-4409-8206-f4f4d4581e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:32.693603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429852e9-49a6-40f7-9ff3-eb23df387982 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:32.733189 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1326/2624] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:32.742711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ec226-c4c0-4a62-9a92-4bc97268734f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:32.742871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d0095-1f33-4409-8206-f4f4d4581e20 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:32.743376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ec226-c4c0-4a62-9a92-4bc97268734f service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:32.743665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ec226-c4c0-4a62-9a92-4bc97268734f service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:32.743947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ec226-c4c0-4a62-9a92-4bc97268734f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:32.744231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-fd1d0095-1f33-4409-8206-f4f4d4581e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:32.745810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-710ec226-c4c0-4a62-9a92-4bc97268734f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:32.763752 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:32.800789 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d0095-1f33-4409-8206-f4f4d4581e20 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:32.804486 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ac62-f93a-41f1-acf8-e555edf56622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:32.809223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ac62-f93a-41f1-acf8-e555edf56622 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:32.809223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ac62-f93a-41f1-acf8-e555edf56622 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:32.809223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ac62-f93a-41f1-acf8-e555edf56622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:32.811632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ec226-c4c0-4a62-9a92-4bc97268734f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:32.813101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ec226-c4c0-4a62-9a92-4bc97268734f service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:32.815535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57ac62-f93a-41f1-acf8-e555edf56622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:32.816958 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57ac62-f93a-41f1-acf8-e555edf56622 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:32.822863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1299/2625] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:32.827258 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1327/2626] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 19:03:33.256369 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ec226-c4c0-4a62-9a92-4bc97268734f service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:33.258590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-710ec226-c4c0-4a62-9a92-4bc97268734f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:33.281793 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ec226-c4c0-4a62-9a92-4bc97268734f service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:33.288411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:33.289415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:33.290253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:33.290528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] PATH_INFO: `/v3/projects/9e69d72ef4584bf5872a2512035dfe11` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:33.292426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e69d72ef4584bf5872a2512035dfe11)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:33.294409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:33.402963 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e69d72ef4584bf5872a2512035dfe11', 'request_id': 'req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:33.487589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e69d72ef4584bf5872a2512035dfe11', 'request_id': 'req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:33.493829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e69d72ef4584bf5872a2512035dfe11', 'request_id': 'req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:33.511772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] The token cache is being invalidate because project 9e69d72ef4584bf5872a2512035dfe11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:33.512086 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:33.512785 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-3ef250d5-8c0e-4e48-a689-59159f1d0f6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:33.514724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1300/2627] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:33 2022] DELETE /identity/v3/projects/9e69d72ef4584bf5872a2512035dfe11 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:33.526667 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf75d84-7d17-40dc-9ee6-fb2758a3f4c7 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:33.527208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf75d84-7d17-40dc-9ee6-fb2758a3f4c7 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:33.527208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf75d84-7d17-40dc-9ee6-fb2758a3f4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:33.534118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:33.547125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9cf75d84-7d17-40dc-9ee6-fb2758a3f4c7 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:33.550430 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9cf75d84-7d17-40dc-9ee6-fb2758a3f4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:33.609820 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1328/2628] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:33.617545 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a3ea7-adf3-4a64-9710-74affacf68ef None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:33.617834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a3ea7-adf3-4a64-9710-74affacf68ef None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:33.618107 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a3ea7-adf3-4a64-9710-74affacf68ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:33.636611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-019a3ea7-adf3-4a64-9710-74affacf68ef None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:33.637867 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-019a3ea7-adf3-4a64-9710-74affacf68ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:33.645804 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1301/2629] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:33.649913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf75d84-7d17-40dc-9ee6-fb2758a3f4c7 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:33.660087 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf75d84-7d17-40dc-9ee6-fb2758a3f4c7 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:33.664659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc8222-4f10-4c7a-9f7b-93d9c530ab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:33.665349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8222-4f10-4c7a-9f7b-93d9c530ab09 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:33.665638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8222-4f10-4c7a-9f7b-93d9c530ab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:33.665917 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8222-4f10-4c7a-9f7b-93d9c530ab09 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:33.666641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc8222-4f10-4c7a-9f7b-93d9c530ab09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:03:33.666952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc8222-4f10-4c7a-9f7b-93d9c530ab09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:33.669316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc8222-4f10-4c7a-9f7b-93d9c530ab09 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:33.680697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1329/2630] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 19:03:33.684900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a3ea7-adf3-4a64-9710-74affacf68ef None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:33.695606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a3ea7-adf3-4a64-9710-74affacf68ef None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:33.700972 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86312790-5e1f-48a9-b8ec-1f82d0ce4ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:33.701665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86312790-5e1f-48a9-b8ec-1f82d0ce4ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:33.701978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86312790-5e1f-48a9-b8ec-1f82d0ce4ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:33.702325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86312790-5e1f-48a9-b8ec-1f82d0ce4ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:33.702992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86312790-5e1f-48a9-b8ec-1f82d0ce4ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:33.704878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86312790-5e1f-48a9-b8ec-1f82d0ce4ae5 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:33.796999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1302/2631] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:33 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:33.801088 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc8222-4f10-4c7a-9f7b-93d9c530ab09 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:33.810987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc8222-4f10-4c7a-9f7b-93d9c530ab09 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:33.867310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6bb17-69c8-4f34-a8d9-55e3686b3a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:33.867899 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6bb17-69c8-4f34-a8d9-55e3686b3a4a admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:33.868184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6bb17-69c8-4f34-a8d9-55e3686b3a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:33.868461 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6bb17-69c8-4f34-a8d9-55e3686b3a4a admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:33.869148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6bb17-69c8-4f34-a8d9-55e3686b3a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:33.871088 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e6bb17-69c8-4f34-a8d9-55e3686b3a4a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:33.973458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1330/2632] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:33 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:33.977566 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86312790-5e1f-48a9-b8ec-1f82d0ce4ae5 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:33.987820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86312790-5e1f-48a9-b8ec-1f82d0ce4ae5 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:33.992950 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c71ec4-2ea8-406b-af29-4b442d6adb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:33.993679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c71ec4-2ea8-406b-af29-4b442d6adb8d admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:33.993970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c71ec4-2ea8-406b-af29-4b442d6adb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:33.994254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c71ec4-2ea8-406b-af29-4b442d6adb8d admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:33.994972 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c71ec4-2ea8-406b-af29-4b442d6adb8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:33.996720 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c71ec4-2ea8-406b-af29-4b442d6adb8d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:34.007410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1303/2633] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:34.011422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6bb17-69c8-4f34-a8d9-55e3686b3a4a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:34.022305 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6bb17-69c8-4f34-a8d9-55e3686b3a4a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:34.029025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fba0b3-58df-48a3-a863-e4c9d61cc1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:34.029025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fba0b3-58df-48a3-a863-e4c9d61cc1dd admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:34.029025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fba0b3-58df-48a3-a863-e4c9d61cc1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:34.029025 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fba0b3-58df-48a3-a863-e4c9d61cc1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:34.029659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fba0b3-58df-48a3-a863-e4c9d61cc1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:34.031244 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fba0b3-58df-48a3-a863-e4c9d61cc1dd admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:34.041091 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1331/2634] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:34.045110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c71ec4-2ea8-406b-af29-4b442d6adb8d admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:34.055165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c71ec4-2ea8-406b-af29-4b442d6adb8d admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:34.060247 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae360306-5b95-47ab-ae76-ceddee85af87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:34.061131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae360306-5b95-47ab-ae76-ceddee85af87 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:34.061429 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae360306-5b95-47ab-ae76-ceddee85af87 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:34.061704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae360306-5b95-47ab-ae76-ceddee85af87 admin admin] PATH_INFO: `/v3/projects/68e5263154724df2969ccfe7418a0ef8/users/2cb4e74879ad47748a167369a603b8f0/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:34.076723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae360306-5b95-47ab-ae76-ceddee85af87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e5263154724df2969ccfe7418a0ef8, user_id=2cb4e74879ad47748a167369a603b8f0, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:34.079387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae360306-5b95-47ab-ae76-ceddee85af87 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:34.185483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1304/2635] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:34 2022] PUT /identity/v3/projects/68e5263154724df2969ccfe7418a0ef8/users/2cb4e74879ad47748a167369a603b8f0/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:34.196543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749cbf0-10fd-456e-acaa-68ce6c2e1232 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:34.196822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749cbf0-10fd-456e-acaa-68ce6c2e1232 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:34.197098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749cbf0-10fd-456e-acaa-68ce6c2e1232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:34.223731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-d749cbf0-10fd-456e-acaa-68ce6c2e1232 None None] MFA Rules not processed for user `2cb4e74879ad47748a167369a603b8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:34.224940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d749cbf0-10fd-456e-acaa-68ce6c2e1232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:34.282923 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:34.288043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1332/2636] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:34.304251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae360306-5b95-47ab-ae76-ceddee85af87 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:34.305971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ae360306-5b95-47ab-ae76-ceddee85af87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:34.341120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:34.362170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae360306-5b95-47ab-ae76-ceddee85af87 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:34.365995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a56e6b-08cd-4ce8-9d38-fa8108228166 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:34.366848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a56e6b-08cd-4ce8-9d38-fa8108228166 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:34.367313 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a56e6b-08cd-4ce8-9d38-fa8108228166 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:34.367955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a56e6b-08cd-4ce8-9d38-fa8108228166 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:34.376980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a56e6b-08cd-4ce8-9d38-fa8108228166 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:34.378502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a56e6b-08cd-4ce8-9d38-fa8108228166 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:34.388593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1305/2637] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:38.548373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749cbf0-10fd-456e-acaa-68ce6c2e1232 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:38.550616 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d749cbf0-10fd-456e-acaa-68ce6c2e1232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:38.613774 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d749cbf0-10fd-456e-acaa-68ce6c2e1232 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:38.618329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:38.619121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:38.619406 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:38.619685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] PATH_INFO: `/v3/users/7f49b5aa62e1466c870d1a2028496a9b` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:38.620864 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f49b5aa62e1466c870d1a2028496a9b)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:38.622797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:38.808686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f49b5aa62e1466c870d1a2028496a9b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:38.809369 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f49b5aa62e1466c870d1a2028496a9b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:38.816259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f49b5aa62e1466c870d1a2028496a9b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:38.816929 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f49b5aa62e1466c870d1a2028496a9b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:38.828535 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f49b5aa62e1466c870d1a2028496a9b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:38.928234 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1333/2638] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:38 2022] DELETE /identity/v3/users/7f49b5aa62e1466c870d1a2028496a9b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:38.932687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdfbe5-b17a-40e6-981b-d6ba140f5f80 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:38.933185 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdfbe5-b17a-40e6-981b-d6ba140f5f80 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:38.933343 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdfbe5-b17a-40e6-981b-d6ba140f5f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:38.950256 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-5efdfbe5-b17a-40e6-981b-d6ba140f5f80 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:38.951480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5efdfbe5-b17a-40e6-981b-d6ba140f5f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:39.011459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1306/2639] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:39.021837 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:39.023392 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:39.081011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c432b3-e8c3-4648-bcc7-dc6a449c2635 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:39.084710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a62e36-239b-4873-ab6e-094ee84f6937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:39.085383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a62e36-239b-4873-ab6e-094ee84f6937 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:39.085672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a62e36-239b-4873-ab6e-094ee84f6937 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:39.085949 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a62e36-239b-4873-ab6e-094ee84f6937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:39.094878 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a62e36-239b-4873-ab6e-094ee84f6937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:39.096938 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a62e36-239b-4873-ab6e-094ee84f6937 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:39.106311 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1334/2640] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:39.500118 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdfbe5-b17a-40e6-981b-d6ba140f5f80 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:39.501558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5efdfbe5-b17a-40e6-981b-d6ba140f5f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:39.514666 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdfbe5-b17a-40e6-981b-d6ba140f5f80 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:39.520096 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:39.520982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:39.521288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:39.521529 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] PATH_INFO: `/v3/projects/99eeaf8e613e4a4c908e3ba52ee900e8` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:39.522778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=99eeaf8e613e4a4c908e3ba52ee900e8)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:39.524710 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:39.643651 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99eeaf8e613e4a4c908e3ba52ee900e8', 'request_id': 'req-2ad44602-2a11-4cf8-b323-921f46068bcd'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:39.699260 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99eeaf8e613e4a4c908e3ba52ee900e8', 'request_id': 'req-2ad44602-2a11-4cf8-b323-921f46068bcd'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:39.706471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99eeaf8e613e4a4c908e3ba52ee900e8', 'request_id': 'req-2ad44602-2a11-4cf8-b323-921f46068bcd'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:39.717436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] The token cache is being invalidate because project 99eeaf8e613e4a4c908e3ba52ee900e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:39.717727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:39.718363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:39.719886 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1307/2641] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:39 2022] DELETE /identity/v3/projects/99eeaf8e613e4a4c908e3ba52ee900e8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:39.723908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a62e36-239b-4873-ab6e-094ee84f6937 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:39.733617 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a62e36-239b-4873-ab6e-094ee84f6937 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:39.750632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:39.792954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b4f05-3b36-45d6-b2c2-a8a5c940c14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:39.793856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b4f05-3b36-45d6-b2c2-a8a5c940c14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:39.794141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b4f05-3b36-45d6-b2c2-a8a5c940c14f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:39.794415 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b4f05-3b36-45d6-b2c2-a8a5c940c14f admin admin] PATH_INFO: `/v3/users/2de97c3d7da04746af777035b7331c5c` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:39.795840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b4f05-3b36-45d6-b2c2-a8a5c940c14f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de97c3d7da04746af777035b7331c5c)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:39.798485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b4f05-3b36-45d6-b2c2-a8a5c940c14f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:39.978709 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-e79b4f05-3b36-45d6-b2c2-a8a5c940c14f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de97c3d7da04746af777035b7331c5c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:39.979385 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-e79b4f05-3b36-45d6-b2c2-a8a5c940c14f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de97c3d7da04746af777035b7331c5c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:39.986912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-e79b4f05-3b36-45d6-b2c2-a8a5c940c14f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de97c3d7da04746af777035b7331c5c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:39.987568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-e79b4f05-3b36-45d6-b2c2-a8a5c940c14f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de97c3d7da04746af777035b7331c5c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:39.999233 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-e79b4f05-3b36-45d6-b2c2-a8a5c940c14f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de97c3d7da04746af777035b7331c5c'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:40.093967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1335/2642] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:39 2022] DELETE /identity/v3/users/2de97c3d7da04746af777035b7331c5c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:40.529595 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:40.531182 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:40.549116 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad44602-2a11-4cf8-b323-921f46068bcd admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:40.608709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:40.609610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:40.609898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:40.615558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] PATH_INFO: `/v3/projects/a315bb61d9a1403fa5d12c5273d7f289` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:40.615558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a315bb61d9a1403fa5d12c5273d7f289)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:40.615558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:40.615558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:40.711956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a315bb61d9a1403fa5d12c5273d7f289', 'request_id': 'req-379acafd-3fbc-4388-930c-4c1fbcb32c26'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:40.779274 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a315bb61d9a1403fa5d12c5273d7f289', 'request_id': 'req-379acafd-3fbc-4388-930c-4c1fbcb32c26'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:40.785346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a315bb61d9a1403fa5d12c5273d7f289', 'request_id': 'req-379acafd-3fbc-4388-930c-4c1fbcb32c26'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:40.798933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] The token cache is being invalidate because project a315bb61d9a1403fa5d12c5273d7f289 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:40.799330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:40.799981 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-379acafd-3fbc-4388-930c-4c1fbcb32c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:40.801455 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1308/2643] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:40 2022] DELETE /identity/v3/projects/a315bb61d9a1403fa5d12c5273d7f289 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:40.811049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a90394-8c07-4fbd-8057-444bfaa937b4 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:40.811528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a90394-8c07-4fbd-8057-444bfaa937b4 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:40.811778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a90394-8c07-4fbd-8057-444bfaa937b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:40.831303 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-41a90394-8c07-4fbd-8057-444bfaa937b4 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:40.832522 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-41a90394-8c07-4fbd-8057-444bfaa937b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:40.890630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1336/2644] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:40.899731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f243b-d21f-4d9e-b9ef-1cad31532882 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:40.900024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f243b-d21f-4d9e-b9ef-1cad31532882 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:40.900319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f243b-d21f-4d9e-b9ef-1cad31532882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:40.919214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d86f243b-d21f-4d9e-b9ef-1cad31532882 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:40.920390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d86f243b-d21f-4d9e-b9ef-1cad31532882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:40.928318 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1309/2645] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:40.932753 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a90394-8c07-4fbd-8057-444bfaa937b4 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:40.943212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a90394-8c07-4fbd-8057-444bfaa937b4 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:40.948007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b56e38-0805-4b67-8740-e35aaf24e255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:40.948679 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b56e38-0805-4b67-8740-e35aaf24e255 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:40.948946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b56e38-0805-4b67-8740-e35aaf24e255 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:40.949221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b56e38-0805-4b67-8740-e35aaf24e255 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:40.949971 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b56e38-0805-4b67-8740-e35aaf24e255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:03:40.950281 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b56e38-0805-4b67-8740-e35aaf24e255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:40.951883 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b56e38-0805-4b67-8740-e35aaf24e255 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:40.964254 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1337/2646] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:03:40.968741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f243b-d21f-4d9e-b9ef-1cad31532882 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:40.980396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f243b-d21f-4d9e-b9ef-1cad31532882 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:40.984645 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dac15-033f-4c83-907f-b6fd616dbcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:40.985344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dac15-033f-4c83-907f-b6fd616dbcd4 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:40.985629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dac15-033f-4c83-907f-b6fd616dbcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:40.985900 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dac15-033f-4c83-907f-b6fd616dbcd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:40.986687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dac15-033f-4c83-907f-b6fd616dbcd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:40.988576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dac15-033f-4c83-907f-b6fd616dbcd4 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:41.061541 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1310/2647] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:40 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:41.066220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b56e38-0805-4b67-8740-e35aaf24e255 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:41.076294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b56e38-0805-4b67-8740-e35aaf24e255 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:41.100522 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:41.132948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5405c2-7166-4e26-8fb1-a6967c09d6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:41.134767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5405c2-7166-4e26-8fb1-a6967c09d6c0 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:41.134767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5405c2-7166-4e26-8fb1-a6967c09d6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:41.134767 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5405c2-7166-4e26-8fb1-a6967c09d6c0 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:41.135061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5405c2-7166-4e26-8fb1-a6967c09d6c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:41.136720 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5405c2-7166-4e26-8fb1-a6967c09d6c0 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:41.214180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1338/2648] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:41 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:41.218283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dac15-033f-4c83-907f-b6fd616dbcd4 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:41.228858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dac15-033f-4c83-907f-b6fd616dbcd4 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:41.233695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90aa40-40e3-46ca-aeaf-f7d877e3c245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:41.234360 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90aa40-40e3-46ca-aeaf-f7d877e3c245 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:41.234652 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90aa40-40e3-46ca-aeaf-f7d877e3c245 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:41.234937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90aa40-40e3-46ca-aeaf-f7d877e3c245 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:41.235700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90aa40-40e3-46ca-aeaf-f7d877e3c245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:41.237894 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90aa40-40e3-46ca-aeaf-f7d877e3c245 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:41.251140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1311/2649] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:41.252979 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5405c2-7166-4e26-8fb1-a6967c09d6c0 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:41.263225 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5405c2-7166-4e26-8fb1-a6967c09d6c0 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:41.267980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0a34e-2f65-498b-9fd6-8b2908590156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:41.268638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0a34e-2f65-498b-9fd6-8b2908590156 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:41.268925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0a34e-2f65-498b-9fd6-8b2908590156 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:41.269205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0a34e-2f65-498b-9fd6-8b2908590156 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:41.269961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0a34e-2f65-498b-9fd6-8b2908590156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:41.271545 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0a34e-2f65-498b-9fd6-8b2908590156 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:41.281511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1339/2650] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:41.285680 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90aa40-40e3-46ca-aeaf-f7d877e3c245 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:41.295631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90aa40-40e3-46ca-aeaf-f7d877e3c245 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:41.300679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9c10e-f982-4798-b4a2-89d0b0584326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:41.301584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9c10e-f982-4798-b4a2-89d0b0584326 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:41.301869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9c10e-f982-4798-b4a2-89d0b0584326 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:41.302146 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9c10e-f982-4798-b4a2-89d0b0584326 admin admin] PATH_INFO: `/v3/projects/8785bee53f9d4ba29febcfc798bb250b/users/612b8ef27a25442e94a6b9a46d48d2c2/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:41.318927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9c10e-f982-4798-b4a2-89d0b0584326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8785bee53f9d4ba29febcfc798bb250b, user_id=612b8ef27a25442e94a6b9a46d48d2c2, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:41.321669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9c10e-f982-4798-b4a2-89d0b0584326 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:41.373828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1312/2651] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:41 2022] PUT /identity/v3/projects/8785bee53f9d4ba29febcfc798bb250b/users/612b8ef27a25442e94a6b9a46d48d2c2/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:41.387066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1e52-adfd-4be1-a73f-4b583fed4b63 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:41.387430 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1e52-adfd-4be1-a73f-4b583fed4b63 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:41.387718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a1e52-adfd-4be1-a73f-4b583fed4b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:41.415880 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-c69a1e52-adfd-4be1-a73f-4b583fed4b63 None None] MFA Rules not processed for user `612b8ef27a25442e94a6b9a46d48d2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:41.417112 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c69a1e52-adfd-4be1-a73f-4b583fed4b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:41.483550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1340/2652] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:41.489140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9c10e-f982-4798-b4a2-89d0b0584326 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:41.552776 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9c10e-f982-4798-b4a2-89d0b0584326 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:41.557818 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abb652-214d-48e9-b035-e175a99c0425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:41.558574 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abb652-214d-48e9-b035-e175a99c0425 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:41.558859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abb652-214d-48e9-b035-e175a99c0425 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:41.559138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abb652-214d-48e9-b035-e175a99c0425 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:41.559829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abb652-214d-48e9-b035-e175a99c0425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:41.561778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abb652-214d-48e9-b035-e175a99c0425 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:41.678692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1313/2653] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:41 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:41.682951 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a1e52-adfd-4be1-a73f-4b583fed4b63 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:41.694017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a1e52-adfd-4be1-a73f-4b583fed4b63 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:41.772511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5f19a-1f35-4d75-bb2d-dbcb70dad89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:41.773187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5f19a-1f35-4d75-bb2d-dbcb70dad89d admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:41.773494 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5f19a-1f35-4d75-bb2d-dbcb70dad89d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:41.773776 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5f19a-1f35-4d75-bb2d-dbcb70dad89d admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:41.774456 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5f19a-1f35-4d75-bb2d-dbcb70dad89d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:41.776358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5f19a-1f35-4d75-bb2d-dbcb70dad89d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:41.861275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1341/2654] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:41 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:41.863695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abb652-214d-48e9-b035-e175a99c0425 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:41.877292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abb652-214d-48e9-b035-e175a99c0425 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:41.882368 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac970d6-064c-4a0c-8b76-4c14fc5ff0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:41.883070 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac970d6-064c-4a0c-8b76-4c14fc5ff0bb admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:41.883355 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac970d6-064c-4a0c-8b76-4c14fc5ff0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:41.894552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac970d6-064c-4a0c-8b76-4c14fc5ff0bb admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:41.894552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac970d6-064c-4a0c-8b76-4c14fc5ff0bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:41.894552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac970d6-064c-4a0c-8b76-4c14fc5ff0bb admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:41.894552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:41.897543 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1314/2655] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:03:41.901482 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5f19a-1f35-4d75-bb2d-dbcb70dad89d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:41.912762 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5f19a-1f35-4d75-bb2d-dbcb70dad89d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:41.917659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d5d53-4ac9-4f1f-85f5-46dd3894ec04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:41.918334 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d5d53-4ac9-4f1f-85f5-46dd3894ec04 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:41.918649 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d5d53-4ac9-4f1f-85f5-46dd3894ec04 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:41.921336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d5d53-4ac9-4f1f-85f5-46dd3894ec04 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:41.921336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d5d53-4ac9-4f1f-85f5-46dd3894ec04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:41.921336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41d5d53-4ac9-4f1f-85f5-46dd3894ec04 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:41.931602 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1342/2656] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:41.935686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac970d6-064c-4a0c-8b76-4c14fc5ff0bb admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:41.945758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac970d6-064c-4a0c-8b76-4c14fc5ff0bb admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:41.950482 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdc715-ac82-4e03-b3e9-32ac5347f8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:41.951415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdc715-ac82-4e03-b3e9-32ac5347f8b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:41.951701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdc715-ac82-4e03-b3e9-32ac5347f8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:41.951984 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdc715-ac82-4e03-b3e9-32ac5347f8b9 admin admin] PATH_INFO: `/v3/projects/0c5ea82f12494625a373f554ffad2c4f/users/380d3f5efbe243c18ba08bd7d56d454b/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:41.968687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdc715-ac82-4e03-b3e9-32ac5347f8b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5ea82f12494625a373f554ffad2c4f, user_id=380d3f5efbe243c18ba08bd7d56d454b, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:41.972316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdc715-ac82-4e03-b3e9-32ac5347f8b9 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:42.031663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1315/2657] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:41 2022] PUT /identity/v3/projects/0c5ea82f12494625a373f554ffad2c4f/users/380d3f5efbe243c18ba08bd7d56d454b/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:42.043061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ea1e2-598f-4e74-ac73-18e92f04e187 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:42.043351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ea1e2-598f-4e74-ac73-18e92f04e187 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:42.043685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ea1e2-598f-4e74-ac73-18e92f04e187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:42.068700 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:42.072740 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-862ea1e2-598f-4e74-ac73-18e92f04e187 None None] MFA Rules not processed for user `380d3f5efbe243c18ba08bd7d56d454b`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:42.073553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-862ea1e2-598f-4e74-ac73-18e92f04e187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:42.139684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1343/2658] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:42.144758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdc715-ac82-4e03-b3e9-32ac5347f8b9 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:42.204772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdc715-ac82-4e03-b3e9-32ac5347f8b9 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:42.210324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6a398-c371-4341-9388-ca7ef0feb6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:42.211032 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6a398-c371-4341-9388-ca7ef0feb6e1 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:42.211331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6a398-c371-4341-9388-ca7ef0feb6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:42.211604 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6a398-c371-4341-9388-ca7ef0feb6e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:42.212285 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6a398-c371-4341-9388-ca7ef0feb6e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:42.214246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6a398-c371-4341-9388-ca7ef0feb6e1 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:42.294387 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1316/2659] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:42 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:42.298412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ea1e2-598f-4e74-ac73-18e92f04e187 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:42.309004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ea1e2-598f-4e74-ac73-18e92f04e187 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:42.370400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6c17a-31d4-419b-83fa-47b49f230a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:42.371090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6c17a-31d4-419b-83fa-47b49f230a35 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:42.371372 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6c17a-31d4-419b-83fa-47b49f230a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:42.371647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6c17a-31d4-419b-83fa-47b49f230a35 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:42.372358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6c17a-31d4-419b-83fa-47b49f230a35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:42.374322 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6c17a-31d4-419b-83fa-47b49f230a35 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:42.476223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1344/2660] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:42 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:42.480095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6a398-c371-4341-9388-ca7ef0feb6e1 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:42.490447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6a398-c371-4341-9388-ca7ef0feb6e1 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:42.495060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d11386-d30d-4244-a0e7-297becf9958c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:42.495740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d11386-d30d-4244-a0e7-297becf9958c admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:42.496021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d11386-d30d-4244-a0e7-297becf9958c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:42.496293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d11386-d30d-4244-a0e7-297becf9958c admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:42.497058 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d11386-d30d-4244-a0e7-297becf9958c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:42.498682 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d11386-d30d-4244-a0e7-297becf9958c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:42.509703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1317/2661] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:42.513582 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6c17a-31d4-419b-83fa-47b49f230a35 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:42.524123 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6c17a-31d4-419b-83fa-47b49f230a35 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:42.528761 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e5709-b2f9-42d4-b799-0af4b429a717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:42.529664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e5709-b2f9-42d4-b799-0af4b429a717 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:42.529952 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e5709-b2f9-42d4-b799-0af4b429a717 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:42.530232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e5709-b2f9-42d4-b799-0af4b429a717 admin admin] PATH_INFO: `/v3/projects/57c25840062640e6aa2df207ba4ddc94/users/903705a692894bc3a776a9adbc7a0468/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:42.544387 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e5709-b2f9-42d4-b799-0af4b429a717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c25840062640e6aa2df207ba4ddc94, user_id=903705a692894bc3a776a9adbc7a0468, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:42.547058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e5709-b2f9-42d4-b799-0af4b429a717 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:42.625161 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1345/2662] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:42 2022] PUT /identity/v3/projects/57c25840062640e6aa2df207ba4ddc94/users/903705a692894bc3a776a9adbc7a0468/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:42.630200 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d11386-d30d-4244-a0e7-297becf9958c admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:42.640059 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d11386-d30d-4244-a0e7-297becf9958c admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:42.695974 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075345d-1443-4c74-8e4e-84a9dd680be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:42.696669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075345d-1443-4c74-8e4e-84a9dd680be7 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:42.696956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075345d-1443-4c74-8e4e-84a9dd680be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:42.697254 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075345d-1443-4c74-8e4e-84a9dd680be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:42.698013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075345d-1443-4c74-8e4e-84a9dd680be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:42.699639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075345d-1443-4c74-8e4e-84a9dd680be7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:42.709331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1318/2663] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:42.713487 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e5709-b2f9-42d4-b799-0af4b429a717 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:42.723782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e5709-b2f9-42d4-b799-0af4b429a717 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:42.728437 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bbfaa-1690-41c1-9632-0038a676c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:42.729271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bbfaa-1690-41c1-9632-0038a676c3ad admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:42.729554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bbfaa-1690-41c1-9632-0038a676c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:42.729822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bbfaa-1690-41c1-9632-0038a676c3ad admin admin] PATH_INFO: `/v3/projects/57c25840062640e6aa2df207ba4ddc94/users/903705a692894bc3a776a9adbc7a0468/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:42.731784 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1bbfaa-1690-41c1-9632-0038a676c3ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c25840062640e6aa2df207ba4ddc94, user_id=903705a692894bc3a776a9adbc7a0468, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:42.734395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1bbfaa-1690-41c1-9632-0038a676c3ad admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:42.745301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1346/2664] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:42 2022] PUT /identity/v3/projects/57c25840062640e6aa2df207ba4ddc94/users/903705a692894bc3a776a9adbc7a0468/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:42.757056 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfad185-a6f4-4def-a9a5-9b84c8b872f2 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:42.757385 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfad185-a6f4-4def-a9a5-9b84c8b872f2 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:42.757668 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfad185-a6f4-4def-a9a5-9b84c8b872f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:42.783115 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-3cfad185-a6f4-4def-a9a5-9b84c8b872f2 None None] MFA Rules not processed for user `903705a692894bc3a776a9adbc7a0468`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:42.784365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-3cfad185-a6f4-4def-a9a5-9b84c8b872f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:42.817524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:42.850246 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1319/2665] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:42.862436 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bbfaa-1690-41c1-9632-0038a676c3ad admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:42.864259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3a1bbfaa-1690-41c1-9632-0038a676c3ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:42.921664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bbfaa-1690-41c1-9632-0038a676c3ad admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:42.926670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f4711-0f92-4903-959e-2cf8e21ff954 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:42.927328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f4711-0f92-4903-959e-2cf8e21ff954 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:42.927738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f4711-0f92-4903-959e-2cf8e21ff954 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:42.927871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f4711-0f92-4903-959e-2cf8e21ff954 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:42.936497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:42.993011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f4711-0f92-4903-959e-2cf8e21ff954 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:42.994525 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f4711-0f92-4903-959e-2cf8e21ff954 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:43.004111 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1347/2666] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:44.998947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfad185-a6f4-4def-a9a5-9b84c8b872f2 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:45.009840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfad185-a6f4-4def-a9a5-9b84c8b872f2 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:45.013410 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81a740-302d-464e-852b-a3a25c146f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:45.019279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81a740-302d-464e-852b-a3a25c146f0a service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:45.019279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81a740-302d-464e-852b-a3a25c146f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:45.019279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81a740-302d-464e-852b-a3a25c146f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:45.072526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81a740-302d-464e-852b-a3a25c146f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:45.074014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81a740-302d-464e-852b-a3a25c146f0a service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:45.083673 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1320/2667] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:45.693107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f4711-0f92-4903-959e-2cf8e21ff954 service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:45.694565 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a64f4711-0f92-4903-959e-2cf8e21ff954 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:45.769422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f4711-0f92-4903-959e-2cf8e21ff954 service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:45.774010 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:45.774871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:45.775157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:45.775434 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] PATH_INFO: `/v3/users/2cb4e74879ad47748a167369a603b8f0` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:45.776604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb4e74879ad47748a167369a603b8f0)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:45.778551 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:45.981853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb4e74879ad47748a167369a603b8f0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:45.982499 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb4e74879ad47748a167369a603b8f0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:45.996694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb4e74879ad47748a167369a603b8f0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:46.001908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb4e74879ad47748a167369a603b8f0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:46.009714 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb4e74879ad47748a167369a603b8f0'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:46.076427 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1348/2668] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:45 2022] DELETE /identity/v3/users/2cb4e74879ad47748a167369a603b8f0 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:46.081171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890b4cd-1ab6-4e2e-866d-c421ca6e86ff None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:46.081756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890b4cd-1ab6-4e2e-866d-c421ca6e86ff None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:46.081997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890b4cd-1ab6-4e2e-866d-c421ca6e86ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:46.100831 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1890b4cd-1ab6-4e2e-866d-c421ca6e86ff None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:46.102063 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1890b4cd-1ab6-4e2e-866d-c421ca6e86ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:46.160826 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1321/2669] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:46.170591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:46.171986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:46.205595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:46.230712 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef57e9-50ba-4bb5-a04f-eb49057980da admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:46.234338 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e2f03-4206-452a-921b-9e3b69f24686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:46.237685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e2f03-4206-452a-921b-9e3b69f24686 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:46.237685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e2f03-4206-452a-921b-9e3b69f24686 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:46.237685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418e2f03-4206-452a-921b-9e3b69f24686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:46.245147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e2f03-4206-452a-921b-9e3b69f24686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:46.246599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418e2f03-4206-452a-921b-9e3b69f24686 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:46.256913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1349/2670] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:46.400832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890b4cd-1ab6-4e2e-866d-c421ca6e86ff None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:46.402298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1890b4cd-1ab6-4e2e-866d-c421ca6e86ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:46.470669 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890b4cd-1ab6-4e2e-866d-c421ca6e86ff None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:46.474535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b383ec3-54a3-4db7-b359-b6f32ce49eb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:46.475189 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383ec3-54a3-4db7-b359-b6f32ce49eb0 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:46.475526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383ec3-54a3-4db7-b359-b6f32ce49eb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:46.475796 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b383ec3-54a3-4db7-b359-b6f32ce49eb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:46.477633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5b383ec3-54a3-4db7-b359-b6f32ce49eb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:46.508051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:46.555426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b383ec3-54a3-4db7-b359-b6f32ce49eb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:46.557152 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b383ec3-54a3-4db7-b359-b6f32ce49eb0 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:46.568576 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1322/2671] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:46.684785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e2f03-4206-452a-921b-9e3b69f24686 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:46.687838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-418e2f03-4206-452a-921b-9e3b69f24686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:46.701577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418e2f03-4206-452a-921b-9e3b69f24686 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:46.717933 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:46.718842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:46.719157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:46.719433 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] PATH_INFO: `/v3/projects/68e5263154724df2969ccfe7418a0ef8` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:46.720615 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e5263154724df2969ccfe7418a0ef8)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:46.722657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:46.845843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e5263154724df2969ccfe7418a0ef8', 'request_id': 'req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:46.899895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e5263154724df2969ccfe7418a0ef8', 'request_id': 'req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:46.905678 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e5263154724df2969ccfe7418a0ef8', 'request_id': 'req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:46.916692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] The token cache is being invalidate because project 68e5263154724df2969ccfe7418a0ef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:46.916977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:46.917608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-1d089c6a-57fa-4eda-aaeb-24d6ffb85245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:46.919108 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1350/2672] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:46 2022] DELETE /identity/v3/projects/68e5263154724df2969ccfe7418a0ef8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:46.924454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b446bc5-cbb3-4aaf-bb0d-6afd034cde3b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:46.924777 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b446bc5-cbb3-4aaf-bb0d-6afd034cde3b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:46.925057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b446bc5-cbb3-4aaf-bb0d-6afd034cde3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:46.942992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-8b446bc5-cbb3-4aaf-bb0d-6afd034cde3b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:46.947092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8b446bc5-cbb3-4aaf-bb0d-6afd034cde3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:47.005730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1323/2673] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:47.013656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe7f46-95f1-4a24-bf5d-65f709a1efa3 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:47.013986 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe7f46-95f1-4a24-bf5d-65f709a1efa3 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:47.014273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe7f46-95f1-4a24-bf5d-65f709a1efa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:47.032759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-00fe7f46-95f1-4a24-bf5d-65f709a1efa3 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:47.034014 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-00fe7f46-95f1-4a24-bf5d-65f709a1efa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:47.042060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1351/2674] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:47.046197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b446bc5-cbb3-4aaf-bb0d-6afd034cde3b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:47.056196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b446bc5-cbb3-4aaf-bb0d-6afd034cde3b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:47.060780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39841a72-eb43-46e9-ac9c-05e29514f27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:47.062143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39841a72-eb43-46e9-ac9c-05e29514f27c admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:47.062143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39841a72-eb43-46e9-ac9c-05e29514f27c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:47.062143 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39841a72-eb43-46e9-ac9c-05e29514f27c admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:47.062723 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39841a72-eb43-46e9-ac9c-05e29514f27c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:03:47.063035 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39841a72-eb43-46e9-ac9c-05e29514f27c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:47.064638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39841a72-eb43-46e9-ac9c-05e29514f27c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:47.076913 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1324/2675] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:03:47.081367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe7f46-95f1-4a24-bf5d-65f709a1efa3 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:47.090981 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe7f46-95f1-4a24-bf5d-65f709a1efa3 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:47.095715 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d06b3-8cc6-4af8-9eee-58fa038d6a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:47.096358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d06b3-8cc6-4af8-9eee-58fa038d6a42 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:47.096648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d06b3-8cc6-4af8-9eee-58fa038d6a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:47.096926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d06b3-8cc6-4af8-9eee-58fa038d6a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:47.097646 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d06b3-8cc6-4af8-9eee-58fa038d6a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:47.099577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d06b3-8cc6-4af8-9eee-58fa038d6a42 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:47.193792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1352/2676] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:47 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:47.197911 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39841a72-eb43-46e9-ac9c-05e29514f27c admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:47.207979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39841a72-eb43-46e9-ac9c-05e29514f27c admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:47.266306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc87216-391d-4ad3-8344-89c11f4acaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:47.266990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc87216-391d-4ad3-8344-89c11f4acaab admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:47.267277 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc87216-391d-4ad3-8344-89c11f4acaab admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:47.267569 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc87216-391d-4ad3-8344-89c11f4acaab admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:47.268250 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc87216-391d-4ad3-8344-89c11f4acaab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:47.270238 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc87216-391d-4ad3-8344-89c11f4acaab admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:47.352805 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1325/2677] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:47 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:47.356415 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d06b3-8cc6-4af8-9eee-58fa038d6a42 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:47.367677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d06b3-8cc6-4af8-9eee-58fa038d6a42 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:47.372395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe0633-b3d0-4914-a5c9-4cf1054f293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:47.373067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe0633-b3d0-4914-a5c9-4cf1054f293e admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:47.373405 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe0633-b3d0-4914-a5c9-4cf1054f293e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:47.373686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe0633-b3d0-4914-a5c9-4cf1054f293e admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:47.374440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe0633-b3d0-4914-a5c9-4cf1054f293e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:47.376039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe0633-b3d0-4914-a5c9-4cf1054f293e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:47.386778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1353/2678] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:47.390740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc87216-391d-4ad3-8344-89c11f4acaab admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:47.400553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc87216-391d-4ad3-8344-89c11f4acaab admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:47.405111 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf6140-55cd-411e-b2c6-8679d1ead36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:47.405799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf6140-55cd-411e-b2c6-8679d1ead36f admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:47.406088 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf6140-55cd-411e-b2c6-8679d1ead36f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:47.406363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf6140-55cd-411e-b2c6-8679d1ead36f admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:47.407072 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf6140-55cd-411e-b2c6-8679d1ead36f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:47.408744 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf6140-55cd-411e-b2c6-8679d1ead36f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:47.418810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1326/2679] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:47.422627 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe0633-b3d0-4914-a5c9-4cf1054f293e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:47.433190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe0633-b3d0-4914-a5c9-4cf1054f293e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:47.437928 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09651f-080b-49c8-85e7-5d701dbfc999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:47.438823 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09651f-080b-49c8-85e7-5d701dbfc999 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:47.439110 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09651f-080b-49c8-85e7-5d701dbfc999 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:47.439390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a09651f-080b-49c8-85e7-5d701dbfc999 admin admin] PATH_INFO: `/v3/projects/f450f6cb6a174dc0ad93134e93fe75cc/users/d5f31565982b4928abc3bb5df8fbaf5b/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:47.453591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09651f-080b-49c8-85e7-5d701dbfc999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f450f6cb6a174dc0ad93134e93fe75cc, user_id=d5f31565982b4928abc3bb5df8fbaf5b, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:47.456247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a09651f-080b-49c8-85e7-5d701dbfc999 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:47.522886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1354/2680] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:47 2022] PUT /identity/v3/projects/f450f6cb6a174dc0ad93134e93fe75cc/users/d5f31565982b4928abc3bb5df8fbaf5b/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:47.534061 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25744b3-ed07-4094-bc3f-e5e09f157841 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:47.534356 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25744b3-ed07-4094-bc3f-e5e09f157841 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:47.534638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25744b3-ed07-4094-bc3f-e5e09f157841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:47.561960 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-e25744b3-ed07-4094-bc3f-e5e09f157841 None None] MFA Rules not processed for user `d5f31565982b4928abc3bb5df8fbaf5b`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:47.563162 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-e25744b3-ed07-4094-bc3f-e5e09f157841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:47.575322 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:47.627396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1327/2681] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:47.638222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09651f-080b-49c8-85e7-5d701dbfc999 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:47.639929 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8a09651f-080b-49c8-85e7-5d701dbfc999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:47.696090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a09651f-080b-49c8-85e7-5d701dbfc999 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:47.699769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aaf58-dd73-4a71-972e-e6fc25b4231c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:47.700409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aaf58-dd73-4a71-972e-e6fc25b4231c service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:47.700697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aaf58-dd73-4a71-972e-e6fc25b4231c service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:47.700974 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aaf58-dd73-4a71-972e-e6fc25b4231c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:47.706171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:47.709687 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aaf58-dd73-4a71-972e-e6fc25b4231c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:47.711118 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aaf58-dd73-4a71-972e-e6fc25b4231c service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:47.719818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1355/2682] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:49.430667 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3b2aa-c4d5-43f4-94fb-159672c5636a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:49.431154 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3b2aa-c4d5-43f4-94fb-159672c5636a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:49.431262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3b2aa-c4d5-43f4-94fb-159672c5636a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:49.448399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f2e3b2aa-c4d5-43f4-94fb-159672c5636a None None] MFA Rules not processed for user `d5f31565982b4928abc3bb5df8fbaf5b`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:49.449646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f2e3b2aa-c4d5-43f4-94fb-159672c5636a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:49.456503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1328/2683] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:51.215261 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aaf58-dd73-4a71-972e-e6fc25b4231c service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:51.217568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-992aaf58-dd73-4a71-972e-e6fc25b4231c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:51.301379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aaf58-dd73-4a71-972e-e6fc25b4231c service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:51.311270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:51.312688 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:51.313119 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:51.313554 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] PATH_INFO: `/v3/users/612b8ef27a25442e94a6b9a46d48d2c2` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:51.316252 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=612b8ef27a25442e94a6b9a46d48d2c2)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:51.321274 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:51.563890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '612b8ef27a25442e94a6b9a46d48d2c2'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:51.564664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '612b8ef27a25442e94a6b9a46d48d2c2'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:51.572236 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '612b8ef27a25442e94a6b9a46d48d2c2'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:51.573675 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '612b8ef27a25442e94a6b9a46d48d2c2'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:51.586573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '612b8ef27a25442e94a6b9a46d48d2c2'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:51.690257 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1356/2684] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:51 2022] DELETE /identity/v3/users/612b8ef27a25442e94a6b9a46d48d2c2 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:51.695167 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8783d92-efea-4e7e-8827-b2a84daef810 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:51.695686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8783d92-efea-4e7e-8827-b2a84daef810 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:51.695872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8783d92-efea-4e7e-8827-b2a84daef810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:51.713068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f8783d92-efea-4e7e-8827-b2a84daef810 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:51.714334 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f8783d92-efea-4e7e-8827-b2a84daef810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:51.783866 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1329/2685] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:51.794199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:51.795963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:51.874120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706f715-95cf-49b1-9a91-e03e48227944 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:51.881098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04718167-ff3c-4c56-bff6-5d73bc86dd61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:51.881802 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04718167-ff3c-4c56-bff6-5d73bc86dd61 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:51.882088 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04718167-ff3c-4c56-bff6-5d73bc86dd61 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:51.882386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04718167-ff3c-4c56-bff6-5d73bc86dd61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:51.894336 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04718167-ff3c-4c56-bff6-5d73bc86dd61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:51.895847 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04718167-ff3c-4c56-bff6-5d73bc86dd61 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:51.905886 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1357/2686] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:52.341970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8783d92-efea-4e7e-8827-b2a84daef810 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:52.343550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f8783d92-efea-4e7e-8827-b2a84daef810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:52.357068 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8783d92-efea-4e7e-8827-b2a84daef810 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:52.362469 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:52.366589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:52.366589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:52.366589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] PATH_INFO: `/v3/projects/8785bee53f9d4ba29febcfc798bb250b` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:52.366589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8785bee53f9d4ba29febcfc798bb250b)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:52.367484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:52.512691 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8785bee53f9d4ba29febcfc798bb250b', 'request_id': 'req-c4282e44-c42b-45ed-8543-187bbce11ea2'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:52.597542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8785bee53f9d4ba29febcfc798bb250b', 'request_id': 'req-c4282e44-c42b-45ed-8543-187bbce11ea2'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:52.606345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8785bee53f9d4ba29febcfc798bb250b', 'request_id': 'req-c4282e44-c42b-45ed-8543-187bbce11ea2'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:52.628513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] The token cache is being invalidate because project 8785bee53f9d4ba29febcfc798bb250b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:52.628897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:52.630207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:52.636142 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1330/2687] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:52 2022] DELETE /identity/v3/projects/8785bee53f9d4ba29febcfc798bb250b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:52.636849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04718167-ff3c-4c56-bff6-5d73bc86dd61 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:52.648125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04718167-ff3c-4c56-bff6-5d73bc86dd61 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:52.659983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:52.713619 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:52.714451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:52.714733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:52.715011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] PATH_INFO: `/v3/users/380d3f5efbe243c18ba08bd7d56d454b` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:52.716224 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] RBAC: Authorizing `identity:delete_user(user_id=380d3f5efbe243c18ba08bd7d56d454b)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:52.718199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:52.956117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '380d3f5efbe243c18ba08bd7d56d454b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:52.957439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '380d3f5efbe243c18ba08bd7d56d454b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:52.968618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '380d3f5efbe243c18ba08bd7d56d454b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:52.969577 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '380d3f5efbe243c18ba08bd7d56d454b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:52.987448 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '380d3f5efbe243c18ba08bd7d56d454b'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:53.067868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1358/2688] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:52 2022] DELETE /identity/v3/users/380d3f5efbe243c18ba08bd7d56d454b => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:53.704769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:53.704769 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:53.719930 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4282e44-c42b-45ed-8543-187bbce11ea2 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:53.766658 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:53.807298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:53.808435 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:53.808930 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:53.809453 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] PATH_INFO: `/v3/projects/0c5ea82f12494625a373f554ffad2c4f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:53.811582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5ea82f12494625a373f554ffad2c4f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:53.813961 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:53.933129 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5ea82f12494625a373f554ffad2c4f', 'request_id': 'req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:53.992209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5ea82f12494625a373f554ffad2c4f', 'request_id': 'req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:53.998527 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5ea82f12494625a373f554ffad2c4f', 'request_id': 'req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:54.010488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] The token cache is being invalidate because project 0c5ea82f12494625a373f554ffad2c4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:54.010780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:54.011542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:54.013272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1331/2689] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:53 2022] DELETE /identity/v3/projects/0c5ea82f12494625a373f554ffad2c4f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:54.017379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:54.018987 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:54.034584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c7242-43c8-44c4-9060-1daf39516549 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:54.100477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8cb40-663a-4725-92b3-0efaca8e7bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:54.101398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8cb40-663a-4725-92b3-0efaca8e7bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:54.101682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8cb40-663a-4725-92b3-0efaca8e7bce admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:54.101955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8cb40-663a-4725-92b3-0efaca8e7bce admin admin] PATH_INFO: `/v3/users/903705a692894bc3a776a9adbc7a0468` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:54.103128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8cb40-663a-4725-92b3-0efaca8e7bce admin admin] RBAC: Authorizing `identity:delete_user(user_id=903705a692894bc3a776a9adbc7a0468)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:54.105250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8cb40-663a-4725-92b3-0efaca8e7bce admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:54.329178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:54.338859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-15c8cb40-663a-4725-92b3-0efaca8e7bce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '903705a692894bc3a776a9adbc7a0468'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:54.339546 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-15c8cb40-663a-4725-92b3-0efaca8e7bce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '903705a692894bc3a776a9adbc7a0468'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:54.346700 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-15c8cb40-663a-4725-92b3-0efaca8e7bce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '903705a692894bc3a776a9adbc7a0468'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:54.347359 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-15c8cb40-663a-4725-92b3-0efaca8e7bce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '903705a692894bc3a776a9adbc7a0468'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:54.358836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-15c8cb40-663a-4725-92b3-0efaca8e7bce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '903705a692894bc3a776a9adbc7a0468'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:54.447076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1359/2690] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:03:54 2022] DELETE /identity/v3/users/903705a692894bc3a776a9adbc7a0468 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:54.857689 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:54.860051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:54.884016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddac73b-1b0f-49eb-ae41-0a1c4e5bc594 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:54.997164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:54.998114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:54.998408 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:54.998694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] PATH_INFO: `/v3/projects/57c25840062640e6aa2df207ba4ddc94` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:55.000717 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c25840062640e6aa2df207ba4ddc94)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:55.006152 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:55.161005 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c25840062640e6aa2df207ba4ddc94', 'request_id': 'req-5d482c51-0336-4b02-8c64-90137acf3b46'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:55.231130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c25840062640e6aa2df207ba4ddc94', 'request_id': 'req-5d482c51-0336-4b02-8c64-90137acf3b46'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:55.237321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c25840062640e6aa2df207ba4ddc94', 'request_id': 'req-5d482c51-0336-4b02-8c64-90137acf3b46'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:55.240439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:55.249811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] The token cache is being invalidate because project 57c25840062640e6aa2df207ba4ddc94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:03:55.250101 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:55.250772 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-5d482c51-0336-4b02-8c64-90137acf3b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:03:55.252311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1332/2691] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:03:54 2022] DELETE /identity/v3/projects/57c25840062640e6aa2df207ba4ddc94 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:55.258202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7440857-1049-43d8-aff4-4da508f00524 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:55.259198 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7440857-1049-43d8-aff4-4da508f00524 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:55.259540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7440857-1049-43d8-aff4-4da508f00524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:55.279284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-e7440857-1049-43d8-aff4-4da508f00524 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:55.280574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e7440857-1049-43d8-aff4-4da508f00524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:55.357270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1360/2692] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:55.369313 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713bd43-c3ed-4dec-a732-e07fe2d3bfd9 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:55.370131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713bd43-c3ed-4dec-a732-e07fe2d3bfd9 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:55.370239 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713bd43-c3ed-4dec-a732-e07fe2d3bfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:55.389491 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d713bd43-c3ed-4dec-a732-e07fe2d3bfd9 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:55.390839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d713bd43-c3ed-4dec-a732-e07fe2d3bfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:55.399367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1333/2693] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:03:55.403610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7440857-1049-43d8-aff4-4da508f00524 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:55.414157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7440857-1049-43d8-aff4-4da508f00524 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:55.419238 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d07d97-a492-442c-afcd-66cc2fc01b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:55.419967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07d97-a492-442c-afcd-66cc2fc01b6e admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:55.420327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07d97-a492-442c-afcd-66cc2fc01b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:55.420694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d07d97-a492-442c-afcd-66cc2fc01b6e admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:55.421549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d07d97-a492-442c-afcd-66cc2fc01b6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:03:55.421933 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d07d97-a492-442c-afcd-66cc2fc01b6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:55.423599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d07d97-a492-442c-afcd-66cc2fc01b6e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:55.435604 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1361/2694] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:03:55.439933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713bd43-c3ed-4dec-a732-e07fe2d3bfd9 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:55.449567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d713bd43-c3ed-4dec-a732-e07fe2d3bfd9 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:55.454362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065938fe-4f3b-4681-a52d-54bf66b179f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:55.455020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065938fe-4f3b-4681-a52d-54bf66b179f7 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:55.458687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065938fe-4f3b-4681-a52d-54bf66b179f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:55.458687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065938fe-4f3b-4681-a52d-54bf66b179f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:55.458687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065938fe-4f3b-4681-a52d-54bf66b179f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:55.458996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065938fe-4f3b-4681-a52d-54bf66b179f7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:55.571104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1334/2695] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:03:55 2022] POST /identity/v3/projects => generated 399 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:55.575342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d07d97-a492-442c-afcd-66cc2fc01b6e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:55.586262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d07d97-a492-442c-afcd-66cc2fc01b6e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:55.643961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd562612-56d0-469c-ac54-449b9f8dee2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:55.644626 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd562612-56d0-469c-ac54-449b9f8dee2a admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:55.644908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd562612-56d0-469c-ac54-449b9f8dee2a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:55.645780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd562612-56d0-469c-ac54-449b9f8dee2a admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:55.646473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd562612-56d0-469c-ac54-449b9f8dee2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:55.648378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd562612-56d0-469c-ac54-449b9f8dee2a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:55.728335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1362/2696] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:03:55 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:03:55.732512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065938fe-4f3b-4681-a52d-54bf66b179f7 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:55.742811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065938fe-4f3b-4681-a52d-54bf66b179f7 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:55.747577 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8df794-a300-44c6-845f-344a60983467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:55.748231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8df794-a300-44c6-845f-344a60983467 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:55.748520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8df794-a300-44c6-845f-344a60983467 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:55.748799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8df794-a300-44c6-845f-344a60983467 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:55.749573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8df794-a300-44c6-845f-344a60983467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:55.751181 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8df794-a300-44c6-845f-344a60983467 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:55.762352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1335/2697] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:55.768444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd562612-56d0-469c-ac54-449b9f8dee2a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:55.779003 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd562612-56d0-469c-ac54-449b9f8dee2a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:55.784455 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72064a35-bda7-46c5-a44d-b461ea1a84a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:55.785598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72064a35-bda7-46c5-a44d-b461ea1a84a3 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:55.788692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72064a35-bda7-46c5-a44d-b461ea1a84a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:55.788692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72064a35-bda7-46c5-a44d-b461ea1a84a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:55.788692 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72064a35-bda7-46c5-a44d-b461ea1a84a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:55.788972 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72064a35-bda7-46c5-a44d-b461ea1a84a3 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:55.799130 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1363/2698] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:03:55.803544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8df794-a300-44c6-845f-344a60983467 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:55.813485 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8df794-a300-44c6-845f-344a60983467 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:55.818532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9eee9d-0df7-405c-a176-221aeb0fc6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:55.819461 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eee9d-0df7-405c-a176-221aeb0fc6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:55.819764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eee9d-0df7-405c-a176-221aeb0fc6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:55.820036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9eee9d-0df7-405c-a176-221aeb0fc6fd admin admin] PATH_INFO: `/v3/projects/a21717251165458da3c366a4df8ae9a9/users/9ee73745921e4f8aa7b5c49588195389/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:55.834933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eee9d-0df7-405c-a176-221aeb0fc6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a21717251165458da3c366a4df8ae9a9, user_id=9ee73745921e4f8aa7b5c49588195389, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:55.837648 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9eee9d-0df7-405c-a176-221aeb0fc6fd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:55.911472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1336/2699] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:03:55 2022] PUT /identity/v3/projects/a21717251165458da3c366a4df8ae9a9/users/9ee73745921e4f8aa7b5c49588195389/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:03:55.921722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec7f96-527b-4748-b906-f195664aa782 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:55.922042 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec7f96-527b-4748-b906-f195664aa782 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:55.922325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec7f96-527b-4748-b906-f195664aa782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:55.948367 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-dcec7f96-527b-4748-b906-f195664aa782 None None] MFA Rules not processed for user `9ee73745921e4f8aa7b5c49588195389`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:03:55.949595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-dcec7f96-527b-4748-b906-f195664aa782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:55.957520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:03:56.019154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1364/2700] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:03:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:03:56.030706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9eee9d-0df7-405c-a176-221aeb0fc6fd admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:56.032477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-be9eee9d-0df7-405c-a176-221aeb0fc6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:56.088731 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9eee9d-0df7-405c-a176-221aeb0fc6fd admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:56.092330 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb780b-5071-4308-b13a-5dae6c2b9639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:56.093047 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb780b-5071-4308-b13a-5dae6c2b9639 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:56.093373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb780b-5071-4308-b13a-5dae6c2b9639 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:56.093656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb780b-5071-4308-b13a-5dae6c2b9639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:56.101944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb780b-5071-4308-b13a-5dae6c2b9639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:56.103375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb780b-5071-4308-b13a-5dae6c2b9639 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:56.112057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1337/2701] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:56.431506 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec7f96-527b-4748-b906-f195664aa782 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:56.433015 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-dcec7f96-527b-4748-b906-f195664aa782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:56.491044 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec7f96-527b-4748-b906-f195664aa782 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:56.494719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4172550-dc3e-451f-a2d4-7f53ab685d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:56.495417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4172550-dc3e-451f-a2d4-7f53ab685d82 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:56.495724 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4172550-dc3e-451f-a2d4-7f53ab685d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:56.495999 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4172550-dc3e-451f-a2d4-7f53ab685d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:56.497809 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c4172550-dc3e-451f-a2d4-7f53ab685d82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:56.560467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4172550-dc3e-451f-a2d4-7f53ab685d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:56.561965 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4172550-dc3e-451f-a2d4-7f53ab685d82 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:56.572180 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1365/2702] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:03:57.034140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb780b-5071-4308-b13a-5dae6c2b9639 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:03:57.035633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-dccb780b-5071-4308-b13a-5dae6c2b9639 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:57.055427 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb780b-5071-4308-b13a-5dae6c2b9639 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:03:57.059373 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86af5aa-adb7-4626-a6b6-75eb7d7b32a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:03:57.060034 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86af5aa-adb7-4626-a6b6-75eb7d7b32a6 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:03:57.060299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86af5aa-adb7-4626-a6b6-75eb7d7b32a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:03:57.060571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86af5aa-adb7-4626-a6b6-75eb7d7b32a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:03:57.062418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c86af5aa-adb7-4626-a6b6-75eb7d7b32a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:03:57.124935 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86af5aa-adb7-4626-a6b6-75eb7d7b32a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:03:57.126439 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86af5aa-adb7-4626-a6b6-75eb7d7b32a6 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:03:57.136908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1338/2703] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:03:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:00.347790 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4172550-dc3e-451f-a2d4-7f53ab685d82 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:00.351583 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c4172550-dc3e-451f-a2d4-7f53ab685d82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:00.423676 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:00.429677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4172550-dc3e-451f-a2d4-7f53ab685d82 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:00.434948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084a173-cceb-49a7-867e-2f1c61b54ba9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:00.435599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084a173-cceb-49a7-867e-2f1c61b54ba9 service placement] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:00.435882 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084a173-cceb-49a7-867e-2f1c61b54ba9 service placement] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:00.436157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084a173-cceb-49a7-867e-2f1c61b54ba9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:00.437989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e084a173-cceb-49a7-867e-2f1c61b54ba9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:00.450630 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084a173-cceb-49a7-867e-2f1c61b54ba9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:00.452204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084a173-cceb-49a7-867e-2f1c61b54ba9 service placement] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:00.462920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1366/2704] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:00.530628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86af5aa-adb7-4626-a6b6-75eb7d7b32a6 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:00.532346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c86af5aa-adb7-4626-a6b6-75eb7d7b32a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:00.548221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86af5aa-adb7-4626-a6b6-75eb7d7b32a6 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:00.553617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62074c36-8a54-4e7b-bd17-c86637a35fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:00.554494 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074c36-8a54-4e7b-bd17-c86637a35fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:00.554788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074c36-8a54-4e7b-bd17-c86637a35fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:00.555069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074c36-8a54-4e7b-bd17-c86637a35fc2 admin admin] PATH_INFO: `/v3/users/2de523abbc7b4f8b972dc97b8a013237` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:00.556252 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62074c36-8a54-4e7b-bd17-c86637a35fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de523abbc7b4f8b972dc97b8a013237)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:00.558211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62074c36-8a54-4e7b-bd17-c86637a35fc2 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:00.656071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:00.778214 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-62074c36-8a54-4e7b-bd17-c86637a35fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de523abbc7b4f8b972dc97b8a013237'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:00.786938 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-62074c36-8a54-4e7b-bd17-c86637a35fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de523abbc7b4f8b972dc97b8a013237'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:00.787671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-62074c36-8a54-4e7b-bd17-c86637a35fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de523abbc7b4f8b972dc97b8a013237'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:00.802890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-62074c36-8a54-4e7b-bd17-c86637a35fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de523abbc7b4f8b972dc97b8a013237'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:00.803655 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-62074c36-8a54-4e7b-bd17-c86637a35fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de523abbc7b4f8b972dc97b8a013237'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:00.920924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1339/2705] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:04:00 2022] DELETE /identity/v3/users/2de523abbc7b4f8b972dc97b8a013237 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:04:01.274595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084a173-cceb-49a7-867e-2f1c61b54ba9 service placement] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:01.290916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084a173-cceb-49a7-867e-2f1c61b54ba9 service placement] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:01.350153 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:01.351024 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:01.351495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:01.351773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] PATH_INFO: `/v3/projects/0b607bd5abec4c72842e250a639f99f1` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:01.352964 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b607bd5abec4c72842e250a639f99f1)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:01.354983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:01.488481 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b607bd5abec4c72842e250a639f99f1', 'request_id': 'req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:01.541072 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b607bd5abec4c72842e250a639f99f1', 'request_id': 'req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:01.547021 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b607bd5abec4c72842e250a639f99f1', 'request_id': 'req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:01.558222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] The token cache is being invalidate because project 0b607bd5abec4c72842e250a639f99f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:04:01.558511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:01.559157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-0bba840d-bcdd-42b3-9b0b-2c95797da7e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:01.560685 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1367/2706] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:04:01 2022] DELETE /identity/v3/projects/0b607bd5abec4c72842e250a639f99f1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:04:01.565896 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee635c5-653a-43ea-9f1b-7df2ebd83935 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:01.566402 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee635c5-653a-43ea-9f1b-7df2ebd83935 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:01.566517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee635c5-653a-43ea-9f1b-7df2ebd83935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:01.585476 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6ee635c5-653a-43ea-9f1b-7df2ebd83935 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:01.586718 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6ee635c5-653a-43ea-9f1b-7df2ebd83935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:01.650959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1340/2707] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:01.660821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26b40-f7f7-4227-908f-bd132dee29f0 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:01.661141 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26b40-f7f7-4227-908f-bd132dee29f0 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:01.661421 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26b40-f7f7-4227-908f-bd132dee29f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:01.679755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-04f26b40-f7f7-4227-908f-bd132dee29f0 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:01.680985 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-04f26b40-f7f7-4227-908f-bd132dee29f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:01.689213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1368/2708] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:04:01.693924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee635c5-653a-43ea-9f1b-7df2ebd83935 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:01.703554 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee635c5-653a-43ea-9f1b-7df2ebd83935 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:01.708709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eca53a-a7bf-4285-840c-ce3931cb1230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:01.709398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eca53a-a7bf-4285-840c-ce3931cb1230 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:01.709692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eca53a-a7bf-4285-840c-ce3931cb1230 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:01.709970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eca53a-a7bf-4285-840c-ce3931cb1230 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:01.710688 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eca53a-a7bf-4285-840c-ce3931cb1230 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:04:01.710998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eca53a-a7bf-4285-840c-ce3931cb1230 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:01.712603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eca53a-a7bf-4285-840c-ce3931cb1230 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:01.723909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1341/2709] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:04:01.728157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f26b40-f7f7-4227-908f-bd132dee29f0 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:01.737444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f26b40-f7f7-4227-908f-bd132dee29f0 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:01.742124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682ade9-950d-432e-809d-792781c26b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:01.742778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682ade9-950d-432e-809d-792781c26b0a admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:01.743063 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682ade9-950d-432e-809d-792781c26b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:01.743339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682ade9-950d-432e-809d-792781c26b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:01.744044 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682ade9-950d-432e-809d-792781c26b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:01.745980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682ade9-950d-432e-809d-792781c26b0a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:01.874318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1369/2710] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:04:01 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:04:01.878657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eca53a-a7bf-4285-840c-ce3931cb1230 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:01.890425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eca53a-a7bf-4285-840c-ce3931cb1230 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:01.948180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9a2be-b7b1-44f3-a227-23d1feeb7455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:01.949013 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9a2be-b7b1-44f3-a227-23d1feeb7455 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:01.949329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9a2be-b7b1-44f3-a227-23d1feeb7455 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:01.949612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9a2be-b7b1-44f3-a227-23d1feeb7455 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:01.950302 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9a2be-b7b1-44f3-a227-23d1feeb7455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:01.952227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9a2be-b7b1-44f3-a227-23d1feeb7455 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:02.064398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1342/2711] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:04:01 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:04:02.065599 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682ade9-950d-432e-809d-792781c26b0a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:02.076568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682ade9-950d-432e-809d-792781c26b0a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:02.081271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5644d3-08f3-4a16-940b-1f426c45d74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:02.081942 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5644d3-08f3-4a16-940b-1f426c45d74b admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:02.082227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5644d3-08f3-4a16-940b-1f426c45d74b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:02.082505 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5644d3-08f3-4a16-940b-1f426c45d74b admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:02.083308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5644d3-08f3-4a16-940b-1f426c45d74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:02.084909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5644d3-08f3-4a16-940b-1f426c45d74b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:02.096473 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1370/2712] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 19:04:02.100509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9a2be-b7b1-44f3-a227-23d1feeb7455 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:02.110339 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9a2be-b7b1-44f3-a227-23d1feeb7455 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:02.115497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421b6b5-b020-46c2-919d-388d85aa6708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:02.116149 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b6b5-b020-46c2-919d-388d85aa6708 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:02.116436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b6b5-b020-46c2-919d-388d85aa6708 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:02.116737 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421b6b5-b020-46c2-919d-388d85aa6708 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:02.117546 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421b6b5-b020-46c2-919d-388d85aa6708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:02.119176 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421b6b5-b020-46c2-919d-388d85aa6708 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:02.132656 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1343/2713] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:04:02.136916 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5644d3-08f3-4a16-940b-1f426c45d74b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:02.147301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5644d3-08f3-4a16-940b-1f426c45d74b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:02.151810 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06618b7-e32c-4288-8d94-c97d4095612f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:02.152683 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06618b7-e32c-4288-8d94-c97d4095612f admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:02.152966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06618b7-e32c-4288-8d94-c97d4095612f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:02.153264 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06618b7-e32c-4288-8d94-c97d4095612f admin admin] PATH_INFO: `/v3/projects/cfa6530fe6dd49d88f41c57406f4cbb6/users/06722257713746f697d1f609c3956e05/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:02.168301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06618b7-e32c-4288-8d94-c97d4095612f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfa6530fe6dd49d88f41c57406f4cbb6, user_id=06722257713746f697d1f609c3956e05, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:02.170944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06618b7-e32c-4288-8d94-c97d4095612f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:02.238808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1371/2714] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:04:02 2022] PUT /identity/v3/projects/cfa6530fe6dd49d88f41c57406f4cbb6/users/06722257713746f697d1f609c3956e05/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:04:02.243949 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a206bdc-bf3f-4a08-87e2-1ce3aaa789b5 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:02.244244 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a206bdc-bf3f-4a08-87e2-1ce3aaa789b5 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:02.244525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a206bdc-bf3f-4a08-87e2-1ce3aaa789b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:02.262693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-2a206bdc-bf3f-4a08-87e2-1ce3aaa789b5 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:02.264177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2a206bdc-bf3f-4a08-87e2-1ce3aaa789b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:02.268128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:02.323956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1344/2715] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:02.333958 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06618b7-e32c-4288-8d94-c97d4095612f admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:02.335891 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a06618b7-e32c-4288-8d94-c97d4095612f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:02.355088 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:02.399034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06618b7-e32c-4288-8d94-c97d4095612f admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:02.402664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b5df4-22bc-4f2d-8d6c-49051cadb98a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:02.403518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b5df4-22bc-4f2d-8d6c-49051cadb98a service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:02.403800 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b5df4-22bc-4f2d-8d6c-49051cadb98a service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:02.404073 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b5df4-22bc-4f2d-8d6c-49051cadb98a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:02.413190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b5df4-22bc-4f2d-8d6c-49051cadb98a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:02.414636 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b5df4-22bc-4f2d-8d6c-49051cadb98a service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:02.424057 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1372/2716] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:03.829183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a206bdc-bf3f-4a08-87e2-1ce3aaa789b5 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:03.830730 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2a206bdc-bf3f-4a08-87e2-1ce3aaa789b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:03.909702 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a206bdc-bf3f-4a08-87e2-1ce3aaa789b5 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:03.914807 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47e5a0-bf34-4016-aade-1c28b10ceadb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:03.915452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47e5a0-bf34-4016-aade-1c28b10ceadb service placement] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:03.915736 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47e5a0-bf34-4016-aade-1c28b10ceadb service placement] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:03.916011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47e5a0-bf34-4016-aade-1c28b10ceadb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:03.917921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1a47e5a0-bf34-4016-aade-1c28b10ceadb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:03.929405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47e5a0-bf34-4016-aade-1c28b10ceadb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:03.930979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47e5a0-bf34-4016-aade-1c28b10ceadb service placement] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:03.941220 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1345/2717] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:06.726047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26748-fecf-43ac-ab88-93ddd558d585 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:06.726543 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26748-fecf-43ac-ab88-93ddd558d585 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:06.726662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b26748-fecf-43ac-ab88-93ddd558d585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:06.746218 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-42b26748-fecf-43ac-ab88-93ddd558d585 None None] MFA Rules not processed for user `1c0e7e9f9d0f409a8820d8bd8dd239cf`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:06.747818 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-42b26748-fecf-43ac-ab88-93ddd558d585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:06.824240 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1373/2718] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:04:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:07.219421 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47e5a0-bf34-4016-aade-1c28b10ceadb service placement] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:07.220917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1a47e5a0-bf34-4016-aade-1c28b10ceadb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:07.232345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47e5a0-bf34-4016-aade-1c28b10ceadb service placement] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:07.236908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc54234-dae3-4c85-a716-2cdd99169fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:07.238480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54234-dae3-4c85-a716-2cdd99169fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:07.238766 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54234-dae3-4c85-a716-2cdd99169fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:07.239048 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc54234-dae3-4c85-a716-2cdd99169fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:07.241694 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8fc54234-dae3-4c85-a716-2cdd99169fb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:07.252180 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc54234-dae3-4c85-a716-2cdd99169fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:07.253654 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc54234-dae3-4c85-a716-2cdd99169fb8 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:07.260887 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b26748-fecf-43ac-ab88-93ddd558d585 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:07.262320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-42b26748-fecf-43ac-ab88-93ddd558d585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:07.270231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1346/2719] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:07.278991 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b26748-fecf-43ac-ab88-93ddd558d585 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:07.283894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:07.284856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:07.289587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:07.289587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] PATH_INFO: `/v3/users/9ee73745921e4f8aa7b5c49588195389` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:07.289587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee73745921e4f8aa7b5c49588195389)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:07.289587 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:07.543151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee73745921e4f8aa7b5c49588195389'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:07.543919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee73745921e4f8aa7b5c49588195389'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:07.559218 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee73745921e4f8aa7b5c49588195389'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:07.559973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee73745921e4f8aa7b5c49588195389'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:07.586132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee73745921e4f8aa7b5c49588195389'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:07.702924 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1374/2720] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:04:07 2022] DELETE /identity/v3/users/9ee73745921e4f8aa7b5c49588195389 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:04:07.708011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd01da-ad8a-4f54-b899-89cf5f958a96 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:07.708996 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd01da-ad8a-4f54-b899-89cf5f958a96 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:07.709513 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd01da-ad8a-4f54-b899-89cf5f958a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:07.733071 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-24fd01da-ad8a-4f54-b899-89cf5f958a96 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:07.734809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-24fd01da-ad8a-4f54-b899-89cf5f958a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:07.836040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1347/2721] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:07.846606 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:07.849518 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:07.928996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:07.960142 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf2b39-4ea8-4a85-a337-fa43dfb9cedc admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:07.964001 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d42e0-3a5b-416b-b8b0-f0578b27500c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:07.964670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d42e0-3a5b-416b-b8b0-f0578b27500c service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:07.964961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d42e0-3a5b-416b-b8b0-f0578b27500c service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:07.965655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d42e0-3a5b-416b-b8b0-f0578b27500c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:07.989038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d42e0-3a5b-416b-b8b0-f0578b27500c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:07.991435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d42e0-3a5b-416b-b8b0-f0578b27500c service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:08.005140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1375/2722] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:08.275225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468a24-9b62-436b-876f-2ece6d2bbc59 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:08.276036 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468a24-9b62-436b-876f-2ece6d2bbc59 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:08.276472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b468a24-9b62-436b-876f-2ece6d2bbc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:08.310895 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-1b468a24-9b62-436b-876f-2ece6d2bbc59 None None] MFA Rules not processed for user `06722257713746f697d1f609c3956e05`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:08.312139 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1b468a24-9b62-436b-876f-2ece6d2bbc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:08.345311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:08.460428 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1348/2723] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:04:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:08.476379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d42e0-3a5b-416b-b8b0-f0578b27500c service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:08.477940 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-711d42e0-3a5b-416b-b8b0-f0578b27500c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:08.596177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d42e0-3a5b-416b-b8b0-f0578b27500c service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:08.605702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa97608-9a61-4cb2-8e85-87d2fabf1718 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:08.607553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa97608-9a61-4cb2-8e85-87d2fabf1718 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:08.607553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa97608-9a61-4cb2-8e85-87d2fabf1718 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:08.607553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa97608-9a61-4cb2-8e85-87d2fabf1718 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:08.617922 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa97608-9a61-4cb2-8e85-87d2fabf1718 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:08.619809 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa97608-9a61-4cb2-8e85-87d2fabf1718 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:08.645446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1376/2724] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:08.767719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b468a24-9b62-436b-876f-2ece6d2bbc59 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:08.769064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1b468a24-9b62-436b-876f-2ece6d2bbc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:08.793533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b468a24-9b62-436b-876f-2ece6d2bbc59 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:08.808995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:08.809915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:08.810207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:08.810985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] PATH_INFO: `/v3/projects/a21717251165458da3c366a4df8ae9a9` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:08.812970 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a21717251165458da3c366a4df8ae9a9)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:08.824132 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:08.979989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a21717251165458da3c366a4df8ae9a9', 'request_id': 'req-51047050-f227-4150-9848-d81c15c26e79'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:09.069029 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a21717251165458da3c366a4df8ae9a9', 'request_id': 'req-51047050-f227-4150-9848-d81c15c26e79'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:09.078831 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a21717251165458da3c366a4df8ae9a9', 'request_id': 'req-51047050-f227-4150-9848-d81c15c26e79'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:09.094147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] The token cache is being invalidate because project a21717251165458da3c366a4df8ae9a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:04:09.094585 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:09.095242 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-51047050-f227-4150-9848-d81c15c26e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:09.097004 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1349/2725] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:04:08 2022] DELETE /identity/v3/projects/a21717251165458da3c366a4df8ae9a9 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:04:11.498116 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c39026-0832-40a1-b511-b0ff8bcfba32 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:11.498656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c39026-0832-40a1-b511-b0ff8bcfba32 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:11.498656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c39026-0832-40a1-b511-b0ff8bcfba32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:11.524799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-96c39026-0832-40a1-b511-b0ff8bcfba32 None None] MFA Rules not processed for user `06722257713746f697d1f609c3956e05`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:11.526187 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-96c39026-0832-40a1-b511-b0ff8bcfba32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:11.598933 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1377/2726] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:04:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:12.381528 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875f3f1-41b9-4113-853e-9e13a5cd14a0 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:12.382912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875f3f1-41b9-4113-853e-9e13a5cd14a0 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:12.383353 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875f3f1-41b9-4113-853e-9e13a5cd14a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:12.437763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-d875f3f1-41b9-4113-853e-9e13a5cd14a0 None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:12.439567 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d875f3f1-41b9-4113-853e-9e13a5cd14a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:12.525853 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1350/2727] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 19:04:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:12.720190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c39026-0832-40a1-b511-b0ff8bcfba32 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:12.723162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-96c39026-0832-40a1-b511-b0ff8bcfba32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:12.834066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c39026-0832-40a1-b511-b0ff8bcfba32 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:12.845080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991b175-95aa-4115-a0d7-41e073d87eb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:12.847571 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991b175-95aa-4115-a0d7-41e073d87eb4 service swift] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:12.847954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991b175-95aa-4115-a0d7-41e073d87eb4 service swift] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:12.848308 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991b175-95aa-4115-a0d7-41e073d87eb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:12.858956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991b175-95aa-4115-a0d7-41e073d87eb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:12.860853 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991b175-95aa-4115-a0d7-41e073d87eb4 service swift] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:12.864988 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=100 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:12.879078 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1378/2728] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:04:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:04:13.865936 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0b82-9cfd-40b6-a17a-43e6a6b537cb None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:13.866573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0b82-9cfd-40b6-a17a-43e6a6b537cb None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:13.866573 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0b82-9cfd-40b6-a17a-43e6a6b537cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:13.887390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-a62f0b82-9cfd-40b6-a17a-43e6a6b537cb None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:13.888721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a62f0b82-9cfd-40b6-a17a-43e6a6b537cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:13.898287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1351/2729] 10.222.0.81 () {58 vars in 967 bytes} [Thu Jun 16 19:04:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:13.912508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991b175-95aa-4115-a0d7-41e073d87eb4 service swift] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:13.922572 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991b175-95aa-4115-a0d7-41e073d87eb4 service swift] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:13.928425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa3159-a39e-440d-a44e-ea22a1799b16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:13.929121 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa3159-a39e-440d-a44e-ea22a1799b16 service swift] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:13.929682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa3159-a39e-440d-a44e-ea22a1799b16 service swift] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:13.929976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa3159-a39e-440d-a44e-ea22a1799b16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:13.938508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa3159-a39e-440d-a44e-ea22a1799b16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:13.940127 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa3159-a39e-440d-a44e-ea22a1799b16 service swift] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:13.949077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1379/2730] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:04:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:04:14.617909 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb408-86e0-4ed9-9456-8106a7199b1b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:14.618204 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb408-86e0-4ed9-9456-8106a7199b1b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:14.618484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddb408-86e0-4ed9-9456-8106a7199b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:14.636230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-80ddb408-86e0-4ed9-9456-8106a7199b1b None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:14.637468 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-80ddb408-86e0-4ed9-9456-8106a7199b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:14.644849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1352/2731] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 19:04:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:14.656873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa3159-a39e-440d-a44e-ea22a1799b16 service swift] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:14.668148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa3159-a39e-440d-a44e-ea22a1799b16 service swift] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:14.673221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ee124-2881-4273-a8b6-fc407cad1617 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:14.673895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ee124-2881-4273-a8b6-fc407cad1617 service swift] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:14.674213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ee124-2881-4273-a8b6-fc407cad1617 service swift] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:14.674493 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ee124-2881-4273-a8b6-fc407cad1617 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:14.683148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ee124-2881-4273-a8b6-fc407cad1617 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:14.684801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ee124-2881-4273-a8b6-fc407cad1617 service swift] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:14.693059 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1380/2732] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:04:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 19:04:15.265093 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664fd76-593f-4314-b660-761060171d02 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:15.265777 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664fd76-593f-4314-b660-761060171d02 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:15.265890 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664fd76-593f-4314-b660-761060171d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:15.285192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-a664fd76-593f-4314-b660-761060171d02 None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:15.286519 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a664fd76-593f-4314-b660-761060171d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:15.294060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1353/2733] 10.222.0.81 () {58 vars in 967 bytes} [Thu Jun 16 19:04:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:15.308075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ee124-2881-4273-a8b6-fc407cad1617 service swift] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:15.319298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ee124-2881-4273-a8b6-fc407cad1617 service swift] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:15.324482 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60753541-d0c1-404f-9c64-a019d792b06c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:15.325158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60753541-d0c1-404f-9c64-a019d792b06c service swift] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:15.325483 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60753541-d0c1-404f-9c64-a019d792b06c service swift] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:15.325775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60753541-d0c1-404f-9c64-a019d792b06c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:15.337271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60753541-d0c1-404f-9c64-a019d792b06c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:15.337271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60753541-d0c1-404f-9c64-a019d792b06c service swift] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:15.346090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1381/2734] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:04:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:04:18.200312 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664fd76-593f-4314-b660-761060171d02 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:18.202406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-a664fd76-593f-4314-b660-761060171d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:18.231215 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:18.269584 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664fd76-593f-4314-b660-761060171d02 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:18.274440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:18.275309 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:18.275591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:18.275864 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] PATH_INFO: `/v3/users/d5f31565982b4928abc3bb5df8fbaf5b` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:18.277165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5f31565982b4928abc3bb5df8fbaf5b)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:18.279201 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:18.492367 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5f31565982b4928abc3bb5df8fbaf5b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:18.499310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5f31565982b4928abc3bb5df8fbaf5b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:18.500633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5f31565982b4928abc3bb5df8fbaf5b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:18.513103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5f31565982b4928abc3bb5df8fbaf5b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:18.513994 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5f31565982b4928abc3bb5df8fbaf5b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:18.604844 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1354/2735] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:04:18 2022] DELETE /identity/v3/users/d5f31565982b4928abc3bb5df8fbaf5b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:04:18.609425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef109a-ea06-40df-a842-7670b0c87f59 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:18.609898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef109a-ea06-40df-a842-7670b0c87f59 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:18.610171 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef109a-ea06-40df-a842-7670b0c87f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:18.628409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-31ef109a-ea06-40df-a842-7670b0c87f59 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:04:18.629773 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-31ef109a-ea06-40df-a842-7670b0c87f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:18.690466 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1382/2736] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:04:18.700823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:18.702261 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:18.762579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97434f4-6a6e-4902-9719-62204da738ba admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:18.764629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbadf1-e95d-4761-a0d3-f22cf19bd646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:18.765322 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbadf1-e95d-4761-a0d3-f22cf19bd646 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:18.765651 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbadf1-e95d-4761-a0d3-f22cf19bd646 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:18.765952 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbadf1-e95d-4761-a0d3-f22cf19bd646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:18.775082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbadf1-e95d-4761-a0d3-f22cf19bd646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:18.776533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbadf1-e95d-4761-a0d3-f22cf19bd646 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:18.786024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1355/2737] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:19.097661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef109a-ea06-40df-a842-7670b0c87f59 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:19.099148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-31ef109a-ea06-40df-a842-7670b0c87f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:19.111798 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef109a-ea06-40df-a842-7670b0c87f59 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:19.117925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:19.118906 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:19.119199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:19.119484 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] PATH_INFO: `/v3/projects/f450f6cb6a174dc0ad93134e93fe75cc` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:19.120699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f450f6cb6a174dc0ad93134e93fe75cc)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:19.122760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:19.235013 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f450f6cb6a174dc0ad93134e93fe75cc', 'request_id': 'req-3c74d8c0-911b-4391-8bfe-12a42418c1b9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:19.295417 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f450f6cb6a174dc0ad93134e93fe75cc', 'request_id': 'req-3c74d8c0-911b-4391-8bfe-12a42418c1b9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:19.301750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f450f6cb6a174dc0ad93134e93fe75cc', 'request_id': 'req-3c74d8c0-911b-4391-8bfe-12a42418c1b9'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:19.316157 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] The token cache is being invalidate because project f450f6cb6a174dc0ad93134e93fe75cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:04:19.316449 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:19.317106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-3c74d8c0-911b-4391-8bfe-12a42418c1b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:04:19.318926 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1383/2738] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:04:19 2022] DELETE /identity/v3/projects/f450f6cb6a174dc0ad93134e93fe75cc => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:04:26.987702 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbadf1-e95d-4761-a0d3-f22cf19bd646 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:04:26.997114 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbadf1-e95d-4761-a0d3-f22cf19bd646 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:04:27.048220 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:27.058159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3ab4f-d37d-4a55-b9e6-95bdb907a3a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:04:27.058921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ab4f-d37d-4a55-b9e6-95bdb907a3a5 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:04:27.059207 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ab4f-d37d-4a55-b9e6-95bdb907a3a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:04:27.059483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3ab4f-d37d-4a55-b9e6-95bdb907a3a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:04:27.061857 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-97c3ab4f-d37d-4a55-b9e6-95bdb907a3a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:04:27.132298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3ab4f-d37d-4a55-b9e6-95bdb907a3a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:04:27.133989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3ab4f-d37d-4a55-b9e6-95bdb907a3a5 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:04:27.144623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1356/2739] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:04:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:04:29.312638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:04:37.295617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=29 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:05:15.411348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54556516-1600-4c46-9d99-265f6b9ce26e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:05:15.411862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54556516-1600-4c46-9d99-265f6b9ce26e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:05:15.411970 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54556516-1600-4c46-9d99-265f6b9ce26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:05:15.430608 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-54556516-1600-4c46-9d99-265f6b9ce26e None None] MFA Rules not processed for user `06722257713746f697d1f609c3956e05`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:05:15.431872 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-54556516-1600-4c46-9d99-265f6b9ce26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:05:15.488936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1384/2740] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:05:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:05:25.481137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=25 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:07.933487 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3ab4f-d37d-4a55-b9e6-95bdb907a3a5 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:07.934988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-97c3ab4f-d37d-4a55-b9e6-95bdb907a3a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:07.946055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3ab4f-d37d-4a55-b9e6-95bdb907a3a5 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:07.950623 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d699b40-5ebd-48be-8f31-8dfdd56c2d72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:07.951222 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699b40-5ebd-48be-8f31-8dfdd56c2d72 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:07.951683 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699b40-5ebd-48be-8f31-8dfdd56c2d72 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:07.951958 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d699b40-5ebd-48be-8f31-8dfdd56c2d72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:07.953765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1d699b40-5ebd-48be-8f31-8dfdd56c2d72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:08.064412 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d699b40-5ebd-48be-8f31-8dfdd56c2d72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:08.065926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d699b40-5ebd-48be-8f31-8dfdd56c2d72 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:08.078976 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1357/2741] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:10.442963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54556516-1600-4c46-9d99-265f6b9ce26e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:10.444450 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-54556516-1600-4c46-9d99-265f6b9ce26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:10.500699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54556516-1600-4c46-9d99-265f6b9ce26e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:10.504205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1030c04-0442-4025-9a15-f4c20e90f727 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:10.504838 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1030c04-0442-4025-9a15-f4c20e90f727 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:10.505125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1030c04-0442-4025-9a15-f4c20e90f727 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:10.505558 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1030c04-0442-4025-9a15-f4c20e90f727 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:10.507325 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e1030c04-0442-4025-9a15-f4c20e90f727 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:10.520445 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1030c04-0442-4025-9a15-f4c20e90f727 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:10.521908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1030c04-0442-4025-9a15-f4c20e90f727 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:10.532665 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1385/2742] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:18.073602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=34 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:20.192365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d699b40-5ebd-48be-8f31-8dfdd56c2d72 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:20.203169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d699b40-5ebd-48be-8f31-8dfdd56c2d72 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:20.207137 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd763-1d44-4098-8fc7-36895f0126a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:20.211870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd763-1d44-4098-8fc7-36895f0126a1 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:20.211870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd763-1d44-4098-8fc7-36895f0126a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:20.211870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd763-1d44-4098-8fc7-36895f0126a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:20.211870 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-c54bd763-1d44-4098-8fc7-36895f0126a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:20.277579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bd763-1d44-4098-8fc7-36895f0126a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:20.282492 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bd763-1d44-4098-8fc7-36895f0126a1 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:20.288622 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1358/2743] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:20.527871 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=26 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:21.405623 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32678c96-b8b1-4488-8180-d0f72bd156e5 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.405921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32678c96-b8b1-4488-8180-d0f72bd156e5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.406205 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32678c96-b8b1-4488-8180-d0f72bd156e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.423547 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-32678c96-b8b1-4488-8180-d0f72bd156e5 None None] MFA Rules not processed for user `06722257713746f697d1f609c3956e05`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:06:21.424755 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-32678c96-b8b1-4488-8180-d0f72bd156e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:21.432128 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1386/2744] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:06:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:06:21.953978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd763-1d44-4098-8fc7-36895f0126a1 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:21.963066 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd763-1d44-4098-8fc7-36895f0126a1 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:21.966692 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca29df-ccc7-477c-83b3-0e73b4e86d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:21.967333 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca29df-ccc7-477c-83b3-0e73b4e86d19 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:21.967967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca29df-ccc7-477c-83b3-0e73b4e86d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:21.968140 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca29df-ccc7-477c-83b3-0e73b4e86d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:21.970014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2fca29df-ccc7-477c-83b3-0e73b4e86d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:21.980219 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca29df-ccc7-477c-83b3-0e73b4e86d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:21.981678 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca29df-ccc7-477c-83b3-0e73b4e86d19 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:21.989923 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1359/2745] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:22.539148 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32678c96-b8b1-4488-8180-d0f72bd156e5 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:22.541332 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-32678c96-b8b1-4488-8180-d0f72bd156e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:22.632655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32678c96-b8b1-4488-8180-d0f72bd156e5 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:22.638457 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd4902-267d-47e2-b16c-882e7d4f64ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:22.639343 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd4902-267d-47e2-b16c-882e7d4f64ed service placement] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:22.639822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd4902-267d-47e2-b16c-882e7d4f64ed service placement] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:22.640105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd4902-267d-47e2-b16c-882e7d4f64ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:22.642433 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-41cd4902-267d-47e2-b16c-882e7d4f64ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:22.655669 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd4902-267d-47e2-b16c-882e7d4f64ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:22.657038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd4902-267d-47e2-b16c-882e7d4f64ed service placement] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:22.673631 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1387/2746] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:25.693065 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca29df-ccc7-477c-83b3-0e73b4e86d19 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:25.710203 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca29df-ccc7-477c-83b3-0e73b4e86d19 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:25.714324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719307ac-0712-45df-9e7a-e39f54de3b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:25.714986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719307ac-0712-45df-9e7a-e39f54de3b76 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:25.715271 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719307ac-0712-45df-9e7a-e39f54de3b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:25.715554 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719307ac-0712-45df-9e7a-e39f54de3b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:25.717602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-719307ac-0712-45df-9e7a-e39f54de3b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:25.729324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719307ac-0712-45df-9e7a-e39f54de3b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:25.730782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719307ac-0712-45df-9e7a-e39f54de3b76 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:25.740671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1360/2747] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:28.724304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd4902-267d-47e2-b16c-882e7d4f64ed service placement] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:28.725807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-41cd4902-267d-47e2-b16c-882e7d4f64ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:28.741262 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd4902-267d-47e2-b16c-882e7d4f64ed service placement] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:28.746425 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:28.746904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:28.747193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:28.747471 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] PATH_INFO: `/v3/users/1c0e7e9f9d0f409a8820d8bd8dd239cf` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:28.748969 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c0e7e9f9d0f409a8820d8bd8dd239cf)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:28.750971 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:28.888273 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c0e7e9f9d0f409a8820d8bd8dd239cf'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:28.888937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c0e7e9f9d0f409a8820d8bd8dd239cf'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:28.904207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c0e7e9f9d0f409a8820d8bd8dd239cf'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:28.904914 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c0e7e9f9d0f409a8820d8bd8dd239cf'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:28.916944 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c0e7e9f9d0f409a8820d8bd8dd239cf'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:28.969515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1388/2748] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:28 2022] DELETE /identity/v3/users/1c0e7e9f9d0f409a8820d8bd8dd239cf => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:29.216876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719307ac-0712-45df-9e7a-e39f54de3b76 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:29.228615 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719307ac-0712-45df-9e7a-e39f54de3b76 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:29.284366 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:29.285270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:29.285564 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:29.285859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] PATH_INFO: `/v3/projects/fc32e3179c534d83b3d4108a38fb9d19` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:29.287016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc32e3179c534d83b3d4108a38fb9d19)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:29.288928 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:29.397164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc32e3179c534d83b3d4108a38fb9d19', 'request_id': 'req-260f4620-b109-4d85-bd90-47acd87a603f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.477979 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc32e3179c534d83b3d4108a38fb9d19', 'request_id': 'req-260f4620-b109-4d85-bd90-47acd87a603f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.484354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc32e3179c534d83b3d4108a38fb9d19', 'request_id': 'req-260f4620-b109-4d85-bd90-47acd87a603f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.491502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:29.496016 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] The token cache is being invalidate because project fc32e3179c534d83b3d4108a38fb9d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:29.496307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.496939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.498755 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1361/2749] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:29 2022] DELETE /identity/v3/projects/fc32e3179c534d83b3d4108a38fb9d19 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:29.502695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:29.504344 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:29.517522 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d8fd7-f92f-4f66-b1d4-efe4f1f65638 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:29.566925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:29.573717 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:29.574585 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:29.574870 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:29.575194 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] PATH_INFO: `/v3/users/c63c1a0699f04a7c86619f3b0c1245ab` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:29.576395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c63c1a0699f04a7c86619f3b0c1245ab)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:29.578346 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:29.777703 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c63c1a0699f04a7c86619f3b0c1245ab'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.778351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c63c1a0699f04a7c86619f3b0c1245ab'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.785061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c63c1a0699f04a7c86619f3b0c1245ab'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.785730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c63c1a0699f04a7c86619f3b0c1245ab'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.797146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c63c1a0699f04a7c86619f3b0c1245ab'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:29.866173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1389/2750] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:06:29 2022] DELETE /identity/v3/users/c63c1a0699f04a7c86619f3b0c1245ab => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:30.114218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:30.115641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:30.134237 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f4620-b109-4d85-bd90-47acd87a603f admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:30.191299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:30.192160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:30.192447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:30.192725 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] PATH_INFO: `/v3/projects/9dd164b80b2d41e4af0a9c90c41928d1` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:30.193956 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dd164b80b2d41e4af0a9c90c41928d1)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:30.195883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:30.305209 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd164b80b2d41e4af0a9c90c41928d1', 'request_id': 'req-64232076-d538-4570-8561-c328f8f442cf'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:30.356992 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd164b80b2d41e4af0a9c90c41928d1', 'request_id': 'req-64232076-d538-4570-8561-c328f8f442cf'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:30.362872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd164b80b2d41e4af0a9c90c41928d1', 'request_id': 'req-64232076-d538-4570-8561-c328f8f442cf'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:30.374782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] The token cache is being invalidate because project 9dd164b80b2d41e4af0a9c90c41928d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:06:30.375069 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:30.375698 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64232076-d538-4570-8561-c328f8f442cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:06:30.377210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1362/2751] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:06:30 2022] DELETE /identity/v3/projects/9dd164b80b2d41e4af0a9c90c41928d1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:06:36.968496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:06:36.970104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:37.044089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9dd49-b43b-4a06-8290-ea144920d0b6 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:06:37.110327 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bd16a-f3ac-4d54-abeb-1404e86a7b87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:06:37.110992 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bd16a-f3ac-4d54-abeb-1404e86a7b87 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:06:37.111282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bd16a-f3ac-4d54-abeb-1404e86a7b87 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:06:37.111563 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8bd16a-f3ac-4d54-abeb-1404e86a7b87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:06:37.113409 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9d8bd16a-f3ac-4d54-abeb-1404e86a7b87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:06:37.126162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bd16a-f3ac-4d54-abeb-1404e86a7b87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:06:37.127769 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8bd16a-f3ac-4d54-abeb-1404e86a7b87 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:06:37.132601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:37.139472 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1390/2752] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:06:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:06:40.371507 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:06:47.232390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:29.738272 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8ad91-a5c8-442d-857b-9a80b9510a4f None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:29.738787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8ad91-a5c8-442d-857b-9a80b9510a4f None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:29.738897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8ad91-a5c8-442d-857b-9a80b9510a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:29.757587 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-eec8ad91-a5c8-442d-857b-9a80b9510a4f None None] MFA Rules not processed for user `06722257713746f697d1f609c3956e05`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:07:29.758797 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-eec8ad91-a5c8-442d-857b-9a80b9510a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:29.812940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1363/2753] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:07:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:07:39.805479 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=25 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:07:41.242246 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bd16a-f3ac-4d54-abeb-1404e86a7b87 service nova] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:07:41.243682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9d8bd16a-f3ac-4d54-abeb-1404e86a7b87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:41.258159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8bd16a-f3ac-4d54-abeb-1404e86a7b87 service nova] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:07:41.263651 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66ab02-dd0b-4a17-92c9-3749aef4e569 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:07:41.265158 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66ab02-dd0b-4a17-92c9-3749aef4e569 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:07:41.265474 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66ab02-dd0b-4a17-92c9-3749aef4e569 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:07:41.265747 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66ab02-dd0b-4a17-92c9-3749aef4e569 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:07:41.267677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ca66ab02-dd0b-4a17-92c9-3749aef4e569 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:07:41.279395 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66ab02-dd0b-4a17-92c9-3749aef4e569 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:07:41.283202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66ab02-dd0b-4a17-92c9-3749aef4e569 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:07:41.291411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1391/2754] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:07:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:07:51.287017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=6 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:44.871152 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8ad91-a5c8-442d-857b-9a80b9510a4f None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:44.872563 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-eec8ad91-a5c8-442d-857b-9a80b9510a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:44.989001 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8ad91-a5c8-442d-857b-9a80b9510a4f None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:44.993874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368f8cb-6c5d-43b3-9b04-36d2353b2ace service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:45.000553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368f8cb-6c5d-43b3-9b04-36d2353b2ace service placement] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:45.000553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368f8cb-6c5d-43b3-9b04-36d2353b2ace service placement] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:45.000553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368f8cb-6c5d-43b3-9b04-36d2353b2ace service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:45.000553 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0368f8cb-6c5d-43b3-9b04-36d2353b2ace service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:45.008784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368f8cb-6c5d-43b3-9b04-36d2353b2ace service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:45.010398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368f8cb-6c5d-43b3-9b04-36d2353b2ace service placement] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:45.020535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1364/2755] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:08:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:08:45.039726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66ab02-dd0b-4a17-92c9-3749aef4e569 service nova] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:45.039726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ca66ab02-dd0b-4a17-92c9-3749aef4e569 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:45.167304 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66ab02-dd0b-4a17-92c9-3749aef4e569 service nova] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:45.172017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fe555-4532-49be-a537-8d5b046ff849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:45.172910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe555-4532-49be-a537-8d5b046ff849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:45.173192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe555-4532-49be-a537-8d5b046ff849 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:45.173503 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fe555-4532-49be-a537-8d5b046ff849 admin admin] PATH_INFO: `/v3/users/06722257713746f697d1f609c3956e05` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:45.174756 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fe555-4532-49be-a537-8d5b046ff849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06722257713746f697d1f609c3956e05)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:45.176652 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fe555-4532-49be-a537-8d5b046ff849 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:45.372797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-928fe555-4532-49be-a537-8d5b046ff849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06722257713746f697d1f609c3956e05'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:45.373477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-928fe555-4532-49be-a537-8d5b046ff849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06722257713746f697d1f609c3956e05'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:45.380074 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-928fe555-4532-49be-a537-8d5b046ff849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06722257713746f697d1f609c3956e05'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:45.380697 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-928fe555-4532-49be-a537-8d5b046ff849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06722257713746f697d1f609c3956e05'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:45.393839 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-928fe555-4532-49be-a537-8d5b046ff849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06722257713746f697d1f609c3956e05'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:45.490335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1392/2756] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:08:45 2022] DELETE /identity/v3/users/06722257713746f697d1f609c3956e05 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:45.732009 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368f8cb-6c5d-43b3-9b04-36d2353b2ace service placement] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:45.744083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368f8cb-6c5d-43b3-9b04-36d2353b2ace service placement] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:45.807503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:45.808362 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:45.808653 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:45.808927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] PATH_INFO: `/v3/projects/cfa6530fe6dd49d88f41c57406f4cbb6` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:45.810131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfa6530fe6dd49d88f41c57406f4cbb6)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:45.812060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:45.908293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:45.911849 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfa6530fe6dd49d88f41c57406f4cbb6', 'request_id': 'req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:45.968741 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfa6530fe6dd49d88f41c57406f4cbb6', 'request_id': 'req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:45.974251 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfa6530fe6dd49d88f41c57406f4cbb6', 'request_id': 'req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:45.984887 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] The token cache is being invalidate because project cfa6530fe6dd49d88f41c57406f4cbb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:08:45.985122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:45.985778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-64cc5bf9-b88e-4bbc-afac-b0e92d7bfb0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:45.987300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1365/2757] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:08:45 2022] DELETE /identity/v3/projects/cfa6530fe6dd49d88f41c57406f4cbb6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:45.992269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b89cf-9828-4302-b12e-f13a726df3d6 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:45.992560 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b89cf-9828-4302-b12e-f13a726df3d6 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:45.992840 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b89cf-9828-4302-b12e-f13a726df3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.014347 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:46.049250 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-e38b89cf-9828-4302-b12e-f13a726df3d6 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:46.050513 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-e38b89cf-9828-4302-b12e-f13a726df3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:46.106901 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1393/2758] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:46.114955 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf2dfe-27c2-45bc-a259-a57a6ed7e6dd None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:46.115177 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf2dfe-27c2-45bc-a259-a57a6ed7e6dd None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:46.115464 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf2dfe-27c2-45bc-a259-a57a6ed7e6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.134799 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-f6cf2dfe-27c2-45bc-a259-a57a6ed7e6dd None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:46.136593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f6cf2dfe-27c2-45bc-a259-a57a6ed7e6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:46.144786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1366/2759] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:46.149052 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b89cf-9828-4302-b12e-f13a726df3d6 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:46.158862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b89cf-9828-4302-b12e-f13a726df3d6 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:46.165495 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6290df-25cc-4881-9b6d-c8c911d58239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:46.165791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6290df-25cc-4881-9b6d-c8c911d58239 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:46.166086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6290df-25cc-4881-9b6d-c8c911d58239 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:46.166393 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6290df-25cc-4881-9b6d-c8c911d58239 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.167132 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6290df-25cc-4881-9b6d-c8c911d58239 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:08:46.167446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6290df-25cc-4881-9b6d-c8c911d58239 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:46.169060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6290df-25cc-4881-9b6d-c8c911d58239 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:46.185366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1394/2760] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:08:46.186283 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf2dfe-27c2-45bc-a259-a57a6ed7e6dd None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:46.195609 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf2dfe-27c2-45bc-a259-a57a6ed7e6dd None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:46.201010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822b117-dd49-48a1-b7b4-b94320ab6f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:46.201699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822b117-dd49-48a1-b7b4-b94320ab6f97 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:46.201988 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822b117-dd49-48a1-b7b4-b94320ab6f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:46.202289 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822b117-dd49-48a1-b7b4-b94320ab6f97 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.202940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822b117-dd49-48a1-b7b4-b94320ab6f97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:46.205323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822b117-dd49-48a1-b7b4-b94320ab6f97 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:46.315381 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1367/2761] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:08:46 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:08:46.319632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6290df-25cc-4881-9b6d-c8c911d58239 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:46.330380 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6290df-25cc-4881-9b6d-c8c911d58239 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:46.387046 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84873649-f994-4af4-be14-5cdac4d7669e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:46.387722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84873649-f994-4af4-be14-5cdac4d7669e admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:46.388009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84873649-f994-4af4-be14-5cdac4d7669e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:46.388298 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84873649-f994-4af4-be14-5cdac4d7669e admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.388975 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84873649-f994-4af4-be14-5cdac4d7669e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:46.391007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84873649-f994-4af4-be14-5cdac4d7669e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:46.473457 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1395/2762] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:08:46 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:08:46.477542 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822b117-dd49-48a1-b7b4-b94320ab6f97 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:46.488404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822b117-dd49-48a1-b7b4-b94320ab6f97 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:46.492922 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94335049-7628-4273-b089-8b0877feb1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:46.493628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94335049-7628-4273-b089-8b0877feb1cd admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:46.493915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94335049-7628-4273-b089-8b0877feb1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:46.494188 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94335049-7628-4273-b089-8b0877feb1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.494898 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94335049-7628-4273-b089-8b0877feb1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:46.497324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94335049-7628-4273-b089-8b0877feb1cd admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:46.507511 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1368/2763] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:08:46.511358 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84873649-f994-4af4-be14-5cdac4d7669e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:46.521528 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84873649-f994-4af4-be14-5cdac4d7669e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:46.526178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b6a86-47bd-417d-a66a-947185d20d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:46.526829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b6a86-47bd-417d-a66a-947185d20d32 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:46.527111 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b6a86-47bd-417d-a66a-947185d20d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:46.527385 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b6a86-47bd-417d-a66a-947185d20d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.528094 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b6a86-47bd-417d-a66a-947185d20d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:46.529699 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b6a86-47bd-417d-a66a-947185d20d32 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:46.539384 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1396/2764] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:08:46.543785 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94335049-7628-4273-b089-8b0877feb1cd admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:46.553364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94335049-7628-4273-b089-8b0877feb1cd admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:46.558502 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7c5fa-a325-4bd2-9658-0b9978103863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:46.559354 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7c5fa-a325-4bd2-9658-0b9978103863 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:46.559635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7c5fa-a325-4bd2-9658-0b9978103863 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:46.559901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f7c5fa-a325-4bd2-9658-0b9978103863 admin admin] PATH_INFO: `/v3/projects/6fd651efc0174a26b555b9f1c79fd565/users/bf78b0cf59e7463f9c946852f05d53de/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.573780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7c5fa-a325-4bd2-9658-0b9978103863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fd651efc0174a26b555b9f1c79fd565, user_id=bf78b0cf59e7463f9c946852f05d53de, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:46.576310 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f7c5fa-a325-4bd2-9658-0b9978103863 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:46.701447 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1369/2765] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:08:46 2022] PUT /identity/v3/projects/6fd651efc0174a26b555b9f1c79fd565/users/bf78b0cf59e7463f9c946852f05d53de/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:46.712220 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f5dc-daa0-43a8-876b-52ac462c7571 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:46.712511 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f5dc-daa0-43a8-876b-52ac462c7571 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:46.712801 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8f5dc-daa0-43a8-876b-52ac462c7571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.739089 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-efd8f5dc-daa0-43a8-876b-52ac462c7571 None None] MFA Rules not processed for user `bf78b0cf59e7463f9c946852f05d53de`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:46.740269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-efd8f5dc-daa0-43a8-876b-52ac462c7571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:46.782280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:46.804647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1397/2766] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:08:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:08:46.809557 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7c5fa-a325-4bd2-9658-0b9978103863 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:46.871848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f7c5fa-a325-4bd2-9658-0b9978103863 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:46.876640 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32483bc-e16d-4a2e-aa26-d790ed859a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:46.877335 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32483bc-e16d-4a2e-aa26-d790ed859a70 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:46.883446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32483bc-e16d-4a2e-aa26-d790ed859a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:46.883446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32483bc-e16d-4a2e-aa26-d790ed859a70 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:46.883446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32483bc-e16d-4a2e-aa26-d790ed859a70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:46.883446 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32483bc-e16d-4a2e-aa26-d790ed859a70 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:47.017458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1370/2767] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:08:46 2022] POST /identity/v3/projects => generated 381 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:08:47.021556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8f5dc-daa0-43a8-876b-52ac462c7571 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:47.032903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8f5dc-daa0-43a8-876b-52ac462c7571 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:47.087573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace782bd-c099-4546-9905-9b460af98dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:47.088232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace782bd-c099-4546-9905-9b460af98dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:47.088515 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace782bd-c099-4546-9905-9b460af98dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:47.089316 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace782bd-c099-4546-9905-9b460af98dd1 admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:47.089993 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace782bd-c099-4546-9905-9b460af98dd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:47.091892 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace782bd-c099-4546-9905-9b460af98dd1 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:47.220524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1398/2768] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:08:47 2022] POST /identity/v3/users => generated 297 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:08:47.224375 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32483bc-e16d-4a2e-aa26-d790ed859a70 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:47.234430 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32483bc-e16d-4a2e-aa26-d790ed859a70 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:47.239165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf45af6-e690-4375-a9bc-d4a662a6e2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:47.239828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf45af6-e690-4375-a9bc-d4a662a6e2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:47.240113 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf45af6-e690-4375-a9bc-d4a662a6e2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:47.240390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf45af6-e690-4375-a9bc-d4a662a6e2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:47.241106 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf45af6-e690-4375-a9bc-d4a662a6e2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:47.242787 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf45af6-e690-4375-a9bc-d4a662a6e2b3 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:47.253619 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1371/2769] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:08:47.257634 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace782bd-c099-4546-9905-9b460af98dd1 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:47.267968 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace782bd-c099-4546-9905-9b460af98dd1 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:47.272921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1292f10-4354-4711-9daa-cd8f2021ea51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:47.273963 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1292f10-4354-4711-9daa-cd8f2021ea51 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:47.274253 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1292f10-4354-4711-9daa-cd8f2021ea51 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:47.274530 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1292f10-4354-4711-9daa-cd8f2021ea51 admin admin] PATH_INFO: `/v3/projects/8b2b08e680534f909545b1418edaaf23/users/d411c1c26c5f4b91b51cf969fa7fe91b/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:47.289666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1292f10-4354-4711-9daa-cd8f2021ea51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2b08e680534f909545b1418edaaf23, user_id=d411c1c26c5f4b91b51cf969fa7fe91b, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:47.292362 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1292f10-4354-4711-9daa-cd8f2021ea51 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:47.382312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1399/2770] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:08:47 2022] PUT /identity/v3/projects/8b2b08e680534f909545b1418edaaf23/users/d411c1c26c5f4b91b51cf969fa7fe91b/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:47.386695 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf45af6-e690-4375-a9bc-d4a662a6e2b3 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:47.396884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf45af6-e690-4375-a9bc-d4a662a6e2b3 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:47.435803 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:47.451646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23627d-950a-4950-8e67-3f2fd039ae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:47.452306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23627d-950a-4950-8e67-3f2fd039ae71 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:47.452589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23627d-950a-4950-8e67-3f2fd039ae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:47.452863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23627d-950a-4950-8e67-3f2fd039ae71 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:47.453631 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23627d-950a-4950-8e67-3f2fd039ae71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:47.455281 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23627d-950a-4950-8e67-3f2fd039ae71 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:47.465219 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1372/2771] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:08:47.469428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1292f10-4354-4711-9daa-cd8f2021ea51 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:47.479771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1292f10-4354-4711-9daa-cd8f2021ea51 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:47.484326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dd44d-0e1e-44e9-87b5-a1a731c0b022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:47.485203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dd44d-0e1e-44e9-87b5-a1a731c0b022 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:47.485524 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dd44d-0e1e-44e9-87b5-a1a731c0b022 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:47.485806 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8dd44d-0e1e-44e9-87b5-a1a731c0b022 admin admin] PATH_INFO: `/v3/projects/8b2b08e680534f909545b1418edaaf23/users/d411c1c26c5f4b91b51cf969fa7fe91b/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:47.487780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dd44d-0e1e-44e9-87b5-a1a731c0b022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2b08e680534f909545b1418edaaf23, user_id=d411c1c26c5f4b91b51cf969fa7fe91b, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:47.490462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8dd44d-0e1e-44e9-87b5-a1a731c0b022 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:47.500912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1400/2772] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:08:47 2022] PUT /identity/v3/projects/8b2b08e680534f909545b1418edaaf23/users/d411c1c26c5f4b91b51cf969fa7fe91b/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:47.511100 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9049-c0cf-463e-99f1-e6ea709bf22d None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:47.511388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9049-c0cf-463e-99f1-e6ea709bf22d None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:47.511664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9049-c0cf-463e-99f1-e6ea709bf22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:47.537999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-132d9049-c0cf-463e-99f1-e6ea709bf22d None None] MFA Rules not processed for user `d411c1c26c5f4b91b51cf969fa7fe91b`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:47.539211 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-132d9049-c0cf-463e-99f1-e6ea709bf22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:47.604632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1373/2773] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:08:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:47.616908 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dd44d-0e1e-44e9-87b5-a1a731c0b022 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:47.618672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9f8dd44d-0e1e-44e9-87b5-a1a731c0b022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:47.676193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8dd44d-0e1e-44e9-87b5-a1a731c0b022 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:47.679816 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac16355-6d34-47b6-a075-ca8913bd36f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:47.685165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac16355-6d34-47b6-a075-ca8913bd36f0 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:47.685165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac16355-6d34-47b6-a075-ca8913bd36f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:47.685165 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac16355-6d34-47b6-a075-ca8913bd36f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:47.690408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac16355-6d34-47b6-a075-ca8913bd36f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:47.691836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac16355-6d34-47b6-a075-ca8913bd36f0 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:47.701618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1401/2774] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:08:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:08:48.715883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9049-c0cf-463e-99f1-e6ea709bf22d None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:48.724947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9049-c0cf-463e-99f1-e6ea709bf22d None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:48.729991 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce208fc-1f98-47d5-a7e6-dcfb0e04e842 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd411c1c26c5f4b91b51cf969fa7fe91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2b08e680534f909545b1418edaaf23', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:48.730636 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce208fc-1f98-47d5-a7e6-dcfb0e04e842 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:48.730924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce208fc-1f98-47d5-a7e6-dcfb0e04e842 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:48.731200 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce208fc-1f98-47d5-a7e6-dcfb0e04e842 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:48.731912 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce208fc-1f98-47d5-a7e6-dcfb0e04e842 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:08:48.732219 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce208fc-1f98-47d5-a7e6-dcfb0e04e842 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:48.733828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce208fc-1f98-47d5-a7e6-dcfb0e04e842 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:48.744932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1374/2775] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:08:48.749843 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac16355-6d34-47b6-a075-ca8913bd36f0 service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:48.758783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac16355-6d34-47b6-a075-ca8913bd36f0 service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:48.763385 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f944f8e-9b4d-4b32-86bb-e7d3e0da6a41 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd411c1c26c5f4b91b51cf969fa7fe91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2b08e680534f909545b1418edaaf23', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:48.764033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f944f8e-9b4d-4b32-86bb-e7d3e0da6a41 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:48.764326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f944f8e-9b4d-4b32-86bb-e7d3e0da6a41 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:48.764610 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f944f8e-9b4d-4b32-86bb-e7d3e0da6a41 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:48.765323 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f944f8e-9b4d-4b32-86bb-e7d3e0da6a41 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:48.767207 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f944f8e-9b4d-4b32-86bb-e7d3e0da6a41 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:48.868591 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1402/2776] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:08:48 2022] POST /identity/v3/projects => generated 355 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:08:48.873552 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce208fc-1f98-47d5-a7e6-dcfb0e04e842 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:48.883442 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce208fc-1f98-47d5-a7e6-dcfb0e04e842 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:48.938425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94ffa7-2dae-4d1f-b044-1cdd65c0b99a tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd411c1c26c5f4b91b51cf969fa7fe91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2b08e680534f909545b1418edaaf23', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:48.939020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94ffa7-2dae-4d1f-b044-1cdd65c0b99a tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:48.939311 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94ffa7-2dae-4d1f-b044-1cdd65c0b99a tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:48.939592 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94ffa7-2dae-4d1f-b044-1cdd65c0b99a tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:48.940299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94ffa7-2dae-4d1f-b044-1cdd65c0b99a tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:08:48.940609 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94ffa7-2dae-4d1f-b044-1cdd65c0b99a tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:48.942235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94ffa7-2dae-4d1f-b044-1cdd65c0b99a tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:48.953119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1375/2777] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:08:48.988102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f944f8e-9b4d-4b32-86bb-e7d3e0da6a41 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:48.998972 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f944f8e-9b4d-4b32-86bb-e7d3e0da6a41 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:49.003584 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd411c1c26c5f4b91b51cf969fa7fe91b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8b2b08e680534f909545b1418edaaf23', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:49.004438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:49.004731 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:49.005009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] PATH_INFO: `/v3/projects/1bf82a267a4943df93b3cfa05f574d05` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:49.006217 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1bf82a267a4943df93b3cfa05f574d05)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:49.008102 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:49.019296 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:49.145798 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf82a267a4943df93b3cfa05f574d05', 'request_id': 'req-fea860dd-d10f-48cc-9d88-9b76b8498895'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:49.227234 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf82a267a4943df93b3cfa05f574d05', 'request_id': 'req-fea860dd-d10f-48cc-9d88-9b76b8498895'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:49.233462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf82a267a4943df93b3cfa05f574d05', 'request_id': 'req-fea860dd-d10f-48cc-9d88-9b76b8498895'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:49.244932 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] The token cache is being invalidate because project 1bf82a267a4943df93b3cfa05f574d05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:08:49.245223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:49.246163 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-fea860dd-d10f-48cc-9d88-9b76b8498895 tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:49.247694 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1403/2778] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:08:48 2022] DELETE /identity/v3/projects/1bf82a267a4943df93b3cfa05f574d05 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:49.817581 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94ffa7-2dae-4d1f-b044-1cdd65c0b99a tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:49.819378 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-1a94ffa7-2dae-4d1f-b044-1cdd65c0b99a tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:49.864798 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=100 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:49.883045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94ffa7-2dae-4d1f-b044-1cdd65c0b99a tempest-VolumeQuotaClassesTest-1602905827 tempest-VolumeQuotaClassesTest-1602905827-project-admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:49.936927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:49.937757 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:49.938045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:49.938321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] PATH_INFO: `/v3/users/bf78b0cf59e7463f9c946852f05d53de` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:49.939590 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf78b0cf59e7463f9c946852f05d53de)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:49.941571 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:50.137357 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf78b0cf59e7463f9c946852f05d53de'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:50.144161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf78b0cf59e7463f9c946852f05d53de'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:50.144851 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf78b0cf59e7463f9c946852f05d53de'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:50.156897 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf78b0cf59e7463f9c946852f05d53de'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:50.157639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf78b0cf59e7463f9c946852f05d53de'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:50.243050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1376/2779] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:08:49 2022] DELETE /identity/v3/users/bf78b0cf59e7463f9c946852f05d53de => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:50.248378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602096-d617-4744-a81d-34470df49d79 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:50.248879 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602096-d617-4744-a81d-34470df49d79 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:50.248992 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b602096-d617-4744-a81d-34470df49d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:50.266159 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-3b602096-d617-4744-a81d-34470df49d79 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:50.267388 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3b602096-d617-4744-a81d-34470df49d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:50.324726 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1404/2780] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:50.334228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:50.335696 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:50.413258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6448e1-f4f2-47dd-aa2d-5ba87a1f693f admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:50.417388 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b0a19-c5b6-47cf-bc08-249455a8f136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:50.418038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b0a19-c5b6-47cf-bc08-249455a8f136 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:50.418324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b0a19-c5b6-47cf-bc08-249455a8f136 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:50.418606 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b0a19-c5b6-47cf-bc08-249455a8f136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:50.424422 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:50.427940 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b0a19-c5b6-47cf-bc08-249455a8f136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:50.429390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b0a19-c5b6-47cf-bc08-249455a8f136 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:50.439306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1377/2781] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:08:50.858366 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b602096-d617-4744-a81d-34470df49d79 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:50.863018 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-3b602096-d617-4744-a81d-34470df49d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:50.873202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b602096-d617-4744-a81d-34470df49d79 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:50.878011 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:50.878876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:50.879162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:50.879441 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] PATH_INFO: `/v3/projects/6fd651efc0174a26b555b9f1c79fd565` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:50.880620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fd651efc0174a26b555b9f1c79fd565)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:50.882620 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:51.033497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fd651efc0174a26b555b9f1c79fd565', 'request_id': 'req-60098a9a-6162-4bb4-a39c-8c78d112a5f6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:51.118684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fd651efc0174a26b555b9f1c79fd565', 'request_id': 'req-60098a9a-6162-4bb4-a39c-8c78d112a5f6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:51.124633 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fd651efc0174a26b555b9f1c79fd565', 'request_id': 'req-60098a9a-6162-4bb4-a39c-8c78d112a5f6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:51.136156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] The token cache is being invalidate because project 6fd651efc0174a26b555b9f1c79fd565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:08:51.136537 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:51.137184 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:51.138750 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1405/2782] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:08:50 2022] DELETE /identity/v3/projects/6fd651efc0174a26b555b9f1c79fd565 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:51.142761 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b0a19-c5b6-47cf-bc08-249455a8f136 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:51.152929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b0a19-c5b6-47cf-bc08-249455a8f136 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:51.208325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eda460-8764-42e6-856a-6db1c32d0e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:51.209191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eda460-8764-42e6-856a-6db1c32d0e28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:51.209512 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eda460-8764-42e6-856a-6db1c32d0e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:51.209792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eda460-8764-42e6-856a-6db1c32d0e28 admin admin] PATH_INFO: `/v3/users/d411c1c26c5f4b91b51cf969fa7fe91b` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:51.210972 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eda460-8764-42e6-856a-6db1c32d0e28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d411c1c26c5f4b91b51cf969fa7fe91b)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:51.212906 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eda460-8764-42e6-856a-6db1c32d0e28 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:51.463626 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-44eda460-8764-42e6-856a-6db1c32d0e28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd411c1c26c5f4b91b51cf969fa7fe91b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:51.470641 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-44eda460-8764-42e6-856a-6db1c32d0e28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd411c1c26c5f4b91b51cf969fa7fe91b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:51.471304 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-44eda460-8764-42e6-856a-6db1c32d0e28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd411c1c26c5f4b91b51cf969fa7fe91b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:51.483021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-44eda460-8764-42e6-856a-6db1c32d0e28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd411c1c26c5f4b91b51cf969fa7fe91b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:51.483676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-44eda460-8764-42e6-856a-6db1c32d0e28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd411c1c26c5f4b91b51cf969fa7fe91b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:51.552917 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1378/2783] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:08:51 2022] DELETE /identity/v3/users/d411c1c26c5f4b91b51cf969fa7fe91b => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:52.069114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:52.070733 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:52.084836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60098a9a-6162-4bb4-a39c-8c78d112a5f6 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:52.104310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:52.142345 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:52.143212 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:52.143497 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:52.143775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] PATH_INFO: `/v3/projects/8b2b08e680534f909545b1418edaaf23` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:52.144977 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2b08e680534f909545b1418edaaf23)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:52.146898 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:52.280060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2b08e680534f909545b1418edaaf23', 'request_id': 'req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:52.370240 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2b08e680534f909545b1418edaaf23', 'request_id': 'req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:52.376688 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2b08e680534f909545b1418edaaf23', 'request_id': 'req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:52.387873 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] The token cache is being invalidate because project 8b2b08e680534f909545b1418edaaf23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:08:52.388160 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:52.388788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-422e87aa-7e5e-4f96-89e7-ce7b9eefc73e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:52.390337 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1406/2784] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:08:52 2022] DELETE /identity/v3/projects/8b2b08e680534f909545b1418edaaf23 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:52.395210 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0377e2-b4cb-419f-91b4-81e5a3dd6161 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:52.395707 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0377e2-b4cb-419f-91b4-81e5a3dd6161 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:52.395810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0377e2-b4cb-419f-91b4-81e5a3dd6161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:52.415559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-ce0377e2-b4cb-419f-91b4-81e5a3dd6161 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:52.416763 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-ce0377e2-b4cb-419f-91b4-81e5a3dd6161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:52.474138 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1379/2785] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:52.481920 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedfe6c-04f9-4e8b-8e5c-b47665162678 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:52.482211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedfe6c-04f9-4e8b-8e5c-b47665162678 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:52.482490 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedfe6c-04f9-4e8b-8e5c-b47665162678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:52.500783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4bedfe6c-04f9-4e8b-8e5c-b47665162678 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:52.502020 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4bedfe6c-04f9-4e8b-8e5c-b47665162678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:52.509831 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1407/2786] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:52.514424 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0377e2-b4cb-419f-91b4-81e5a3dd6161 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:52.523915 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0377e2-b4cb-419f-91b4-81e5a3dd6161 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:52.528985 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aefaf-2d61-4e08-a923-b552f40f7280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:52.529660 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aefaf-2d61-4e08-a923-b552f40f7280 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:52.529943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aefaf-2d61-4e08-a923-b552f40f7280 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:52.530217 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aefaf-2d61-4e08-a923-b552f40f7280 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:52.530929 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aefaf-2d61-4e08-a923-b552f40f7280 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:08:52.531233 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aefaf-2d61-4e08-a923-b552f40f7280 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:52.532831 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aefaf-2d61-4e08-a923-b552f40f7280 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:52.540937 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:52.544400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1380/2787] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:08:52.548280 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedfe6c-04f9-4e8b-8e5c-b47665162678 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:52.557458 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedfe6c-04f9-4e8b-8e5c-b47665162678 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:52.562131 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda63d7-1ecc-4bd8-957e-25c0dfed5b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:52.562751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda63d7-1ecc-4bd8-957e-25c0dfed5b1e admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:52.563038 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda63d7-1ecc-4bd8-957e-25c0dfed5b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:52.563310 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda63d7-1ecc-4bd8-957e-25c0dfed5b1e admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:52.563997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda63d7-1ecc-4bd8-957e-25c0dfed5b1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:52.565956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda63d7-1ecc-4bd8-957e-25c0dfed5b1e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:52.656821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1408/2788] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:08:52 2022] POST /identity/v3/projects => generated 393 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:08:52.660749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aefaf-2d61-4e08-a923-b552f40f7280 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:52.671135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aefaf-2d61-4e08-a923-b552f40f7280 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:52.725602 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0521c61-5cc1-4cac-8555-3470feb5d929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:52.726273 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0521c61-5cc1-4cac-8555-3470feb5d929 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:52.726558 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0521c61-5cc1-4cac-8555-3470feb5d929 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:52.726841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0521c61-5cc1-4cac-8555-3470feb5d929 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:52.727550 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0521c61-5cc1-4cac-8555-3470feb5d929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:52.729483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0521c61-5cc1-4cac-8555-3470feb5d929 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:52.828729 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1381/2789] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:08:52 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:08:52.832919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda63d7-1ecc-4bd8-957e-25c0dfed5b1e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:52.843657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda63d7-1ecc-4bd8-957e-25c0dfed5b1e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:52.848423 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b226bb2-4df2-4dbc-b063-e84d575cd8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:52.850752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b226bb2-4df2-4dbc-b063-e84d575cd8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:52.850752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b226bb2-4df2-4dbc-b063-e84d575cd8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:52.850752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b226bb2-4df2-4dbc-b063-e84d575cd8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:52.850752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b226bb2-4df2-4dbc-b063-e84d575cd8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:52.852101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b226bb2-4df2-4dbc-b063-e84d575cd8d5 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:52.862462 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1409/2790] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:08:52.866499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0521c61-5cc1-4cac-8555-3470feb5d929 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:52.876987 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0521c61-5cc1-4cac-8555-3470feb5d929 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:52.885169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96178b1-2481-4f31-9995-b343ff490514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:52.885169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96178b1-2481-4f31-9995-b343ff490514 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:52.885169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96178b1-2481-4f31-9995-b343ff490514 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:52.885169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96178b1-2481-4f31-9995-b343ff490514 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:52.885608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96178b1-2481-4f31-9995-b343ff490514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:52.885608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96178b1-2481-4f31-9995-b343ff490514 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:52.895559 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1382/2791] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:08:52.899534 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b226bb2-4df2-4dbc-b063-e84d575cd8d5 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:52.909657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b226bb2-4df2-4dbc-b063-e84d575cd8d5 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:52.914291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fa2a4-57e7-42e0-a3bb-fba43cb31a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:52.919564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fa2a4-57e7-42e0-a3bb-fba43cb31a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:52.919564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fa2a4-57e7-42e0-a3bb-fba43cb31a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:52.919564 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fa2a4-57e7-42e0-a3bb-fba43cb31a0a admin admin] PATH_INFO: `/v3/projects/cceb33eb51e84b4c91088fb85e3ab9be/users/ccec0890256d4ca290bac5fd9dc78997/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:52.930500 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fa2a4-57e7-42e0-a3bb-fba43cb31a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cceb33eb51e84b4c91088fb85e3ab9be, user_id=ccec0890256d4ca290bac5fd9dc78997, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:52.933804 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fa2a4-57e7-42e0-a3bb-fba43cb31a0a admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:53.037559 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1410/2792] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:08:52 2022] PUT /identity/v3/projects/cceb33eb51e84b4c91088fb85e3ab9be/users/ccec0890256d4ca290bac5fd9dc78997/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:53.053878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63785eae-2462-4530-a1f8-3afa3ffc0e0b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:53.054174 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63785eae-2462-4530-a1f8-3afa3ffc0e0b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:53.054459 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63785eae-2462-4530-a1f8-3afa3ffc0e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:53.082098 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-63785eae-2462-4530-a1f8-3afa3ffc0e0b None None] MFA Rules not processed for user `ccec0890256d4ca290bac5fd9dc78997`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:53.083328 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-63785eae-2462-4530-a1f8-3afa3ffc0e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:53.147535 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1383/2793] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:08:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:53.151973 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fa2a4-57e7-42e0-a3bb-fba43cb31a0a admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:53.199913 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:53.212618 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fa2a4-57e7-42e0-a3bb-fba43cb31a0a admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:53.217349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66d36d-915c-469e-b5c8-ee4ff7879159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:53.218007 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66d36d-915c-469e-b5c8-ee4ff7879159 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:53.218324 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66d36d-915c-469e-b5c8-ee4ff7879159 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:53.218605 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66d36d-915c-469e-b5c8-ee4ff7879159 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:53.219348 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66d36d-915c-469e-b5c8-ee4ff7879159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:53.221269 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66d36d-915c-469e-b5c8-ee4ff7879159 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:53.320140 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1411/2794] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:08:53 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:08:53.324646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63785eae-2462-4530-a1f8-3afa3ffc0e0b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:53.335302 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63785eae-2462-4530-a1f8-3afa3ffc0e0b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:53.392158 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ca51c-191e-48b8-8b19-897648e3e057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:53.392829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ca51c-191e-48b8-8b19-897648e3e057 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:53.393117 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ca51c-191e-48b8-8b19-897648e3e057 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:53.393431 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ca51c-191e-48b8-8b19-897648e3e057 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:53.394142 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ca51c-191e-48b8-8b19-897648e3e057 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:53.396064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ca51c-191e-48b8-8b19-897648e3e057 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:53.462624 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:53.469021 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1384/2795] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:08:53 2022] POST /identity/v3/users => generated 302 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:08:53.473120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66d36d-915c-469e-b5c8-ee4ff7879159 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:53.483411 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66d36d-915c-469e-b5c8-ee4ff7879159 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:53.488232 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb64f1-d5bd-4571-9da6-4e77f492e092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:53.488889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb64f1-d5bd-4571-9da6-4e77f492e092 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:53.489178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb64f1-d5bd-4571-9da6-4e77f492e092 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:53.489489 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb64f1-d5bd-4571-9da6-4e77f492e092 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:53.490214 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb64f1-d5bd-4571-9da6-4e77f492e092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:53.491819 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb64f1-d5bd-4571-9da6-4e77f492e092 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:53.503414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1412/2796] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:08:53.507432 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ca51c-191e-48b8-8b19-897648e3e057 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:53.517657 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ca51c-191e-48b8-8b19-897648e3e057 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:53.522220 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b317c9-5eeb-48a6-8d9d-247aea9959e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:53.523094 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b317c9-5eeb-48a6-8d9d-247aea9959e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:53.523383 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b317c9-5eeb-48a6-8d9d-247aea9959e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:53.523664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b317c9-5eeb-48a6-8d9d-247aea9959e0 admin admin] PATH_INFO: `/v3/projects/104e91986f92426c87dc862b346e3c6e/users/4065f396810b4cbf94840cb1b2644287/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:53.538995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b317c9-5eeb-48a6-8d9d-247aea9959e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=104e91986f92426c87dc862b346e3c6e, user_id=4065f396810b4cbf94840cb1b2644287, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:53.541685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b317c9-5eeb-48a6-8d9d-247aea9959e0 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:53.671822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1385/2797] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:08:53 2022] PUT /identity/v3/projects/104e91986f92426c87dc862b346e3c6e/users/4065f396810b4cbf94840cb1b2644287/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:53.675710 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb64f1-d5bd-4571-9da6-4e77f492e092 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:53.686267 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb64f1-d5bd-4571-9da6-4e77f492e092 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:53.740781 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f377990-f52b-4497-a08a-ceaa9d18d0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:53.741399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f377990-f52b-4497-a08a-ceaa9d18d0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:53.741681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f377990-f52b-4497-a08a-ceaa9d18d0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:53.741954 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f377990-f52b-4497-a08a-ceaa9d18d0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:53.742661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f377990-f52b-4497-a08a-ceaa9d18d0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:53.744231 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f377990-f52b-4497-a08a-ceaa9d18d0b7 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:53.754186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1413/2798] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:08:53.758316 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b317c9-5eeb-48a6-8d9d-247aea9959e0 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:53.768942 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b317c9-5eeb-48a6-8d9d-247aea9959e0 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:53.773482 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb09b43-73fa-4f13-9181-d103c405dd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:53.774346 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb09b43-73fa-4f13-9181-d103c405dd03 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:53.774624 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb09b43-73fa-4f13-9181-d103c405dd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:53.774894 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb09b43-73fa-4f13-9181-d103c405dd03 admin admin] PATH_INFO: `/v3/projects/104e91986f92426c87dc862b346e3c6e/users/4065f396810b4cbf94840cb1b2644287/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:53.776884 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb09b43-73fa-4f13-9181-d103c405dd03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=104e91986f92426c87dc862b346e3c6e, user_id=4065f396810b4cbf94840cb1b2644287, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:53.779509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb09b43-73fa-4f13-9181-d103c405dd03 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:53.789473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1386/2799] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:08:53 2022] PUT /identity/v3/projects/104e91986f92426c87dc862b346e3c6e/users/4065f396810b4cbf94840cb1b2644287/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:53.799578 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846d4dd-b6bf-4975-b256-d4f93ad721d5 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:53.804957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846d4dd-b6bf-4975-b256-d4f93ad721d5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:53.804957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846d4dd-b6bf-4975-b256-d4f93ad721d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:53.826828 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9846d4dd-b6bf-4975-b256-d4f93ad721d5 None None] MFA Rules not processed for user `4065f396810b4cbf94840cb1b2644287`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:53.828033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9846d4dd-b6bf-4975-b256-d4f93ad721d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:53.894783 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1414/2800] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:08:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:08:53.906057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb09b43-73fa-4f13-9181-d103c405dd03 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:53.907721 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-7eb09b43-73fa-4f13-9181-d103c405dd03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:53.962934 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb09b43-73fa-4f13-9181-d103c405dd03 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:53.966601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb2c8a-a936-4778-8034-aad35f883190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:53.967242 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb2c8a-a936-4778-8034-aad35f883190 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:53.967534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb2c8a-a936-4778-8034-aad35f883190 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:53.967809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb2c8a-a936-4778-8034-aad35f883190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:53.976635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb2c8a-a936-4778-8034-aad35f883190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:53.978097 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb2c8a-a936-4778-8034-aad35f883190 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:53.987879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1387/2801] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:08:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:08:54.031771 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846d4dd-b6bf-4975-b256-d4f93ad721d5 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:54.033162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9846d4dd-b6bf-4975-b256-d4f93ad721d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:54.045701 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846d4dd-b6bf-4975-b256-d4f93ad721d5 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:54.049204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39281e00-090c-4c5e-a5a3-6e4a09101196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:54.049863 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281e00-090c-4c5e-a5a3-6e4a09101196 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:54.050143 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281e00-090c-4c5e-a5a3-6e4a09101196 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:54.050415 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39281e00-090c-4c5e-a5a3-6e4a09101196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:54.090775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:54.106072 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39281e00-090c-4c5e-a5a3-6e4a09101196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:54.107106 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39281e00-090c-4c5e-a5a3-6e4a09101196 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:54.115956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1415/2802] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:08:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:08:58.801128 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb2c8a-a936-4778-8034-aad35f883190 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:58.861754 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb2c8a-a936-4778-8034-aad35f883190 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:58.866190 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:58.867040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:58.867321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:58.867593 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] PATH_INFO: `/v3/users/ccec0890256d4ca290bac5fd9dc78997` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:58.868740 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccec0890256d4ca290bac5fd9dc78997)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:58.870944 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:59.147704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccec0890256d4ca290bac5fd9dc78997'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:59.155007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccec0890256d4ca290bac5fd9dc78997'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:59.155671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccec0890256d4ca290bac5fd9dc78997'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:59.162440 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:08:59.168057 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccec0890256d4ca290bac5fd9dc78997'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:59.168684 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccec0890256d4ca290bac5fd9dc78997'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:08:59.347295 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1388/2803] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:08:58 2022] DELETE /identity/v3/users/ccec0890256d4ca290bac5fd9dc78997 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:08:59.351821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059042fe-2bf3-4e1b-9fef-4665ca016005 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:59.352320 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059042fe-2bf3-4e1b-9fef-4665ca016005 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:59.352428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059042fe-2bf3-4e1b-9fef-4665ca016005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:59.371594 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-059042fe-2bf3-4e1b-9fef-4665ca016005 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:08:59.372841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-059042fe-2bf3-4e1b-9fef-4665ca016005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:59.430894 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1416/2804] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:08:59.441092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:59.442503 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:59.497813 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c52f67-c32c-428c-9a33-cc963949c861 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:59.502197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b45d4-fbe9-4f20-8a5c-b2b60191531d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:59.502840 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b45d4-fbe9-4f20-8a5c-b2b60191531d service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:59.503125 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b45d4-fbe9-4f20-8a5c-b2b60191531d service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:59.503398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b45d4-fbe9-4f20-8a5c-b2b60191531d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:59.512231 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b45d4-fbe9-4f20-8a5c-b2b60191531d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:08:59.513699 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b45d4-fbe9-4f20-8a5c-b2b60191531d service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:08:59.522854 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1389/2805] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:08:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:08:59.977782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059042fe-2bf3-4e1b-9fef-4665ca016005 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:08:59.979236 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-059042fe-2bf3-4e1b-9fef-4665ca016005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:08:59.992017 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059042fe-2bf3-4e1b-9fef-4665ca016005 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:08:59.996982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:08:59.997903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:08:59.998188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:08:59.998465 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] PATH_INFO: `/v3/projects/cceb33eb51e84b4c91088fb85e3ab9be` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:08:59.999655 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cceb33eb51e84b4c91088fb85e3ab9be)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:00.001595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:00.137659 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cceb33eb51e84b4c91088fb85e3ab9be', 'request_id': 'req-4c7b0228-1849-45a1-99fa-198d0bdf4417'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:00.221702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cceb33eb51e84b4c91088fb85e3ab9be', 'request_id': 'req-4c7b0228-1849-45a1-99fa-198d0bdf4417'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:00.227447 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cceb33eb51e84b4c91088fb85e3ab9be', 'request_id': 'req-4c7b0228-1849-45a1-99fa-198d0bdf4417'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:00.238566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] The token cache is being invalidate because project cceb33eb51e84b4c91088fb85e3ab9be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:09:00.238856 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:00.239491 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:00.241059 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1417/2806] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:08:59 2022] DELETE /identity/v3/projects/cceb33eb51e84b4c91088fb85e3ab9be => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:00.245055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b45d4-fbe9-4f20-8a5c-b2b60191531d service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:00.254875 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b45d4-fbe9-4f20-8a5c-b2b60191531d service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:00.310253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baec86e-c064-44d9-931c-e1d9eeaa1d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:00.311120 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baec86e-c064-44d9-931c-e1d9eeaa1d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:00.311403 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baec86e-c064-44d9-931c-e1d9eeaa1d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:00.311674 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baec86e-c064-44d9-931c-e1d9eeaa1d26 admin admin] PATH_INFO: `/v3/users/4065f396810b4cbf94840cb1b2644287` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:00.312837 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baec86e-c064-44d9-931c-e1d9eeaa1d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4065f396810b4cbf94840cb1b2644287)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:00.314856 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baec86e-c064-44d9-931c-e1d9eeaa1d26 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:00.576458 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8baec86e-c064-44d9-931c-e1d9eeaa1d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4065f396810b4cbf94840cb1b2644287'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:00.583276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8baec86e-c064-44d9-931c-e1d9eeaa1d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4065f396810b4cbf94840cb1b2644287'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:00.584808 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8baec86e-c064-44d9-931c-e1d9eeaa1d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4065f396810b4cbf94840cb1b2644287'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:00.595662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8baec86e-c064-44d9-931c-e1d9eeaa1d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4065f396810b4cbf94840cb1b2644287'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:00.596376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8baec86e-c064-44d9-931c-e1d9eeaa1d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4065f396810b4cbf94840cb1b2644287'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:00.682532 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1390/2807] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:09:00 2022] DELETE /identity/v3/users/4065f396810b4cbf94840cb1b2644287 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:01.204695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:01.206176 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:01.219891 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b0228-1849-45a1-99fa-198d0bdf4417 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:01.278370 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:01.279268 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:01.279555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:01.279833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] PATH_INFO: `/v3/projects/104e91986f92426c87dc862b346e3c6e` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:01.281002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=104e91986f92426c87dc862b346e3c6e)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:01.283033 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:01.290891 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:01.421077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '104e91986f92426c87dc862b346e3c6e', 'request_id': 'req-13276678-3938-4fcc-8989-8026345679b3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:01.501598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '104e91986f92426c87dc862b346e3c6e', 'request_id': 'req-13276678-3938-4fcc-8989-8026345679b3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:01.507208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '104e91986f92426c87dc862b346e3c6e', 'request_id': 'req-13276678-3938-4fcc-8989-8026345679b3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:01.519272 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] The token cache is being invalidate because project 104e91986f92426c87dc862b346e3c6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:09:01.519562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:01.520196 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-13276678-3938-4fcc-8989-8026345679b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:01.521760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1418/2808] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:09:01 2022] DELETE /identity/v3/projects/104e91986f92426c87dc862b346e3c6e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:01.526685 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546da1e0-de63-4725-b7fb-5c42b68faa2b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:01.527141 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546da1e0-de63-4725-b7fb-5c42b68faa2b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:01.527259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546da1e0-de63-4725-b7fb-5c42b68faa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:01.534329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:01.545612 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-546da1e0-de63-4725-b7fb-5c42b68faa2b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:01.556324 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-546da1e0-de63-4725-b7fb-5c42b68faa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:01.602839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1391/2809] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:01.611841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38de510-8610-4188-833f-06f624e5e65e None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:01.612155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38de510-8610-4188-833f-06f624e5e65e None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:01.612431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38de510-8610-4188-833f-06f624e5e65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:01.630850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-a38de510-8610-4188-833f-06f624e5e65e None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:01.632156 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-a38de510-8610-4188-833f-06f624e5e65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:01.640540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1419/2810] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:01.644753 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546da1e0-de63-4725-b7fb-5c42b68faa2b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:01.655127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546da1e0-de63-4725-b7fb-5c42b68faa2b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:01.659628 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5612b56-b724-4431-aa14-524aa93292f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:01.660266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5612b56-b724-4431-aa14-524aa93292f5 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:01.660548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5612b56-b724-4431-aa14-524aa93292f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:01.660820 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5612b56-b724-4431-aa14-524aa93292f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:01.661544 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5612b56-b724-4431-aa14-524aa93292f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:09:01.661848 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5612b56-b724-4431-aa14-524aa93292f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:01.663438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5612b56-b724-4431-aa14-524aa93292f5 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:01.675089 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1392/2811] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:09:01.679203 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38de510-8610-4188-833f-06f624e5e65e None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:01.688328 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38de510-8610-4188-833f-06f624e5e65e None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:01.692890 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e13a8-70b7-44e5-bacf-0d3750a3afde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:01.693562 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e13a8-70b7-44e5-bacf-0d3750a3afde admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:01.693844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e13a8-70b7-44e5-bacf-0d3750a3afde admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:01.694117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e13a8-70b7-44e5-bacf-0d3750a3afde admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:01.694799 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e13a8-70b7-44e5-bacf-0d3750a3afde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:01.696684 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e13a8-70b7-44e5-bacf-0d3750a3afde admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:01.789096 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1420/2812] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:09:01 2022] POST /identity/v3/projects => generated 407 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:09:01.792908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5612b56-b724-4431-aa14-524aa93292f5 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:01.802792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5612b56-b724-4431-aa14-524aa93292f5 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:01.857027 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c3e35-e576-41d2-bf2d-24af1c7bfb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:01.857709 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c3e35-e576-41d2-bf2d-24af1c7bfb19 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:01.857997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c3e35-e576-41d2-bf2d-24af1c7bfb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:01.858276 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c3e35-e576-41d2-bf2d-24af1c7bfb19 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:01.858947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c3e35-e576-41d2-bf2d-24af1c7bfb19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:01.860859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c3e35-e576-41d2-bf2d-24af1c7bfb19 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:01.970681 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1393/2813] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:09:01 2022] POST /identity/v3/users => generated 304 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:09:01.974803 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e13a8-70b7-44e5-bacf-0d3750a3afde admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:01.985019 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e13a8-70b7-44e5-bacf-0d3750a3afde admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:01.989661 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dddf20-ad5d-4fcc-acb4-84f07a59f4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:01.990386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dddf20-ad5d-4fcc-acb4-84f07a59f4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:01.990674 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dddf20-ad5d-4fcc-acb4-84f07a59f4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:01.990950 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dddf20-ad5d-4fcc-acb4-84f07a59f4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:01.991666 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dddf20-ad5d-4fcc-acb4-84f07a59f4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:01.993291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dddf20-ad5d-4fcc-acb4-84f07a59f4b0 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:02.004251 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1421/2814] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:02.008364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c3e35-e576-41d2-bf2d-24af1c7bfb19 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:02.019171 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c3e35-e576-41d2-bf2d-24af1c7bfb19 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:02.023801 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663edd7-b355-4135-9548-21a48f52a1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:02.024477 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663edd7-b355-4135-9548-21a48f52a1f7 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:02.024767 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663edd7-b355-4135-9548-21a48f52a1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:02.025050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663edd7-b355-4135-9548-21a48f52a1f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:02.025810 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663edd7-b355-4135-9548-21a48f52a1f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:02.027419 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663edd7-b355-4135-9548-21a48f52a1f7 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:02.037064 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1394/2815] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:02.040932 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dddf20-ad5d-4fcc-acb4-84f07a59f4b0 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:02.050523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dddf20-ad5d-4fcc-acb4-84f07a59f4b0 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:02.054997 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192df532-d2b3-45be-acd1-cec0bfa437f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:02.055850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192df532-d2b3-45be-acd1-cec0bfa437f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:02.056129 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192df532-d2b3-45be-acd1-cec0bfa437f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:02.056400 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192df532-d2b3-45be-acd1-cec0bfa437f4 admin admin] PATH_INFO: `/v3/projects/7a8aca012120435788feb66ce0b090bc/users/0411b80c5ba24b2e91e3138262866287/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:02.071817 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192df532-d2b3-45be-acd1-cec0bfa437f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8aca012120435788feb66ce0b090bc, user_id=0411b80c5ba24b2e91e3138262866287, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:02.075043 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192df532-d2b3-45be-acd1-cec0bfa437f4 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:02.169895 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1422/2816] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:09:02 2022] PUT /identity/v3/projects/7a8aca012120435788feb66ce0b090bc/users/0411b80c5ba24b2e91e3138262866287/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:02.180112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998f35a-8310-4d11-aecc-ffe4edc39eea None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:02.180406 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998f35a-8310-4d11-aecc-ffe4edc39eea None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:02.180943 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998f35a-8310-4d11-aecc-ffe4edc39eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:02.207306 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-4998f35a-8310-4d11-aecc-ffe4edc39eea None None] MFA Rules not processed for user `0411b80c5ba24b2e91e3138262866287`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:02.208499 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-4998f35a-8310-4d11-aecc-ffe4edc39eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:02.268331 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:02.271997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1395/2817] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:02 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:02.276437 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192df532-d2b3-45be-acd1-cec0bfa437f4 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:02.336076 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192df532-d2b3-45be-acd1-cec0bfa437f4 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:02.340813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5ccfe-ccd1-4644-8d8f-f448c1a71e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:02.341502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5ccfe-ccd1-4644-8d8f-f448c1a71e74 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:02.341792 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5ccfe-ccd1-4644-8d8f-f448c1a71e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:02.342070 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5ccfe-ccd1-4644-8d8f-f448c1a71e74 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:02.342760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5ccfe-ccd1-4644-8d8f-f448c1a71e74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:02.344642 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5ccfe-ccd1-4644-8d8f-f448c1a71e74 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:02.445842 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1423/2818] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:09:02 2022] POST /identity/v3/projects => generated 409 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:09:02.449934 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998f35a-8310-4d11-aecc-ffe4edc39eea None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:02.460436 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998f35a-8310-4d11-aecc-ffe4edc39eea None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:02.515679 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6f8e8-809d-4359-8a28-6dc1e2b04d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:02.516347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6f8e8-809d-4359-8a28-6dc1e2b04d82 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:02.516633 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6f8e8-809d-4359-8a28-6dc1e2b04d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:02.516914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a6f8e8-809d-4359-8a28-6dc1e2b04d82 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:02.517635 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6f8e8-809d-4359-8a28-6dc1e2b04d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:02.519538 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a6f8e8-809d-4359-8a28-6dc1e2b04d82 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:02.644420 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1396/2819] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:09:02 2022] POST /identity/v3/users => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:09:02.648476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5ccfe-ccd1-4644-8d8f-f448c1a71e74 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:02.659523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5ccfe-ccd1-4644-8d8f-f448c1a71e74 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:02.664047 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d4301-604e-4e98-a136-871481cf0836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:02.664698 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d4301-604e-4e98-a136-871481cf0836 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:02.664983 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d4301-604e-4e98-a136-871481cf0836 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:02.665301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d4301-604e-4e98-a136-871481cf0836 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:02.666031 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d4301-604e-4e98-a136-871481cf0836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:02.668233 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d4301-604e-4e98-a136-871481cf0836 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:02.678927 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1424/2820] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:02.682980 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6f8e8-809d-4359-8a28-6dc1e2b04d82 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:02.692727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a6f8e8-809d-4359-8a28-6dc1e2b04d82 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:02.697351 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc8fd0-e8a6-48aa-ae02-95bcacf52f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:02.698218 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc8fd0-e8a6-48aa-ae02-95bcacf52f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:02.698526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc8fd0-e8a6-48aa-ae02-95bcacf52f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:02.698823 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc8fd0-e8a6-48aa-ae02-95bcacf52f81 admin admin] PATH_INFO: `/v3/projects/5ca47d311a784ec8bdc3df26d8c984eb/users/fa5cff3ad3514badb5a5bc1834fddda3/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:02.713084 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc8fd0-e8a6-48aa-ae02-95bcacf52f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca47d311a784ec8bdc3df26d8c984eb, user_id=fa5cff3ad3514badb5a5bc1834fddda3, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:02.716279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc8fd0-e8a6-48aa-ae02-95bcacf52f81 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:02.793756 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1397/2821] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:09:02 2022] PUT /identity/v3/projects/5ca47d311a784ec8bdc3df26d8c984eb/users/fa5cff3ad3514badb5a5bc1834fddda3/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 19:09:02.797780 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d4301-604e-4e98-a136-871481cf0836 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:02.807825 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d4301-604e-4e98-a136-871481cf0836 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:02.811399 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:02.863351 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5a75b-28e3-450b-be5e-84685875675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:02.864004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5a75b-28e3-450b-be5e-84685875675f admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:02.864286 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5a75b-28e3-450b-be5e-84685875675f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:02.864557 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5a75b-28e3-450b-be5e-84685875675f admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:02.865386 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5a75b-28e3-450b-be5e-84685875675f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:02.867029 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5a75b-28e3-450b-be5e-84685875675f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:02.876681 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1425/2822] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:02.880921 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc8fd0-e8a6-48aa-ae02-95bcacf52f81 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:02.891136 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc8fd0-e8a6-48aa-ae02-95bcacf52f81 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:02.895745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481ac11-10c3-468f-8348-0acda8686e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:02.896556 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481ac11-10c3-468f-8348-0acda8686e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:02.896841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481ac11-10c3-468f-8348-0acda8686e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:02.897119 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481ac11-10c3-468f-8348-0acda8686e4e admin admin] PATH_INFO: `/v3/projects/5ca47d311a784ec8bdc3df26d8c984eb/users/fa5cff3ad3514badb5a5bc1834fddda3/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:02.899075 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481ac11-10c3-468f-8348-0acda8686e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca47d311a784ec8bdc3df26d8c984eb, user_id=fa5cff3ad3514badb5a5bc1834fddda3, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:02.902258 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481ac11-10c3-468f-8348-0acda8686e4e admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:02.912250 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1398/2823] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:09:02 2022] PUT /identity/v3/projects/5ca47d311a784ec8bdc3df26d8c984eb/users/fa5cff3ad3514badb5a5bc1834fddda3/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:02.922907 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522e14c-4915-4589-8db5-76a0b70ad045 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:02.923197 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522e14c-4915-4589-8db5-76a0b70ad045 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:02.923477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522e14c-4915-4589-8db5-76a0b70ad045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:02.949357 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-7522e14c-4915-4589-8db5-76a0b70ad045 None None] MFA Rules not processed for user `fa5cff3ad3514badb5a5bc1834fddda3`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:02.950553 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7522e14c-4915-4589-8db5-76a0b70ad045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:03.016722 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1426/2824] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:02 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:03.027711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481ac11-10c3-468f-8348-0acda8686e4e admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:03.029399 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6481ac11-10c3-468f-8348-0acda8686e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:03.083703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481ac11-10c3-468f-8348-0acda8686e4e admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:03.087379 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19311b9c-f99b-4acb-8de1-a2d10dfd1e19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:03.088015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19311b9c-f99b-4acb-8de1-a2d10dfd1e19 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:03.088296 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19311b9c-f99b-4acb-8de1-a2d10dfd1e19 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:03.088569 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19311b9c-f99b-4acb-8de1-a2d10dfd1e19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:03.098345 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19311b9c-f99b-4acb-8de1-a2d10dfd1e19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:03.099775 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19311b9c-f99b-4acb-8de1-a2d10dfd1e19 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:03.109456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1399/2825] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:03.553408 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522e14c-4915-4589-8db5-76a0b70ad045 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:03.563125 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522e14c-4915-4589-8db5-76a0b70ad045 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:03.566762 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b837d8-0cff-4d38-b57a-d904460d4569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:03.567401 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b837d8-0cff-4d38-b57a-d904460d4569 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:03.567689 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b837d8-0cff-4d38-b57a-d904460d4569 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:03.567967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b837d8-0cff-4d38-b57a-d904460d4569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:03.626889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b837d8-0cff-4d38-b57a-d904460d4569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:03.628335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b837d8-0cff-4d38-b57a-d904460d4569 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:03.637297 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1427/2826] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:03 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:10.820597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19311b9c-f99b-4acb-8de1-a2d10dfd1e19 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:10.822031 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-19311b9c-f99b-4acb-8de1-a2d10dfd1e19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:10.882227 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:10.913700 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19311b9c-f99b-4acb-8de1-a2d10dfd1e19 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:10.918764 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647808c-254b-46a3-860f-f74d0dd5de09 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:10.919411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647808c-254b-46a3-860f-f74d0dd5de09 service placement] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:10.919701 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647808c-254b-46a3-860f-f74d0dd5de09 service placement] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:10.919983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647808c-254b-46a3-860f-f74d0dd5de09 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:10.921773 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-5647808c-254b-46a3-860f-f74d0dd5de09 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:10.933426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647808c-254b-46a3-860f-f74d0dd5de09 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:10.935079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647808c-254b-46a3-860f-f74d0dd5de09 service placement] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:10.946517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1400/2827] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:13.633230 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=88 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:13.759211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b837d8-0cff-4d38-b57a-d904460d4569 service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:13.821444 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b837d8-0cff-4d38-b57a-d904460d4569 service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:13.826036 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:13.826904 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:13.827193 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:13.827477 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] PATH_INFO: `/v3/users/0411b80c5ba24b2e91e3138262866287` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:13.828760 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0411b80c5ba24b2e91e3138262866287)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:13.830712 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:14.057561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0411b80c5ba24b2e91e3138262866287'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:14.058208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0411b80c5ba24b2e91e3138262866287'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:14.064847 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0411b80c5ba24b2e91e3138262866287'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:14.065492 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0411b80c5ba24b2e91e3138262866287'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:14.077909 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0411b80c5ba24b2e91e3138262866287'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:14.167953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1428/2828] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:09:13 2022] DELETE /identity/v3/users/0411b80c5ba24b2e91e3138262866287 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:14.172380 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b52e1-fdf3-4981-93e9-9796ddd80a5b None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:14.172832 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b52e1-fdf3-4981-93e9-9796ddd80a5b None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:14.172963 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b52e1-fdf3-4981-93e9-9796ddd80a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:14.190092 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-3b5b52e1-fdf3-4981-93e9-9796ddd80a5b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:14.191292 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-3b5b52e1-fdf3-4981-93e9-9796ddd80a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:14.249438 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1401/2829] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:14.259695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:14.261312 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:14.317318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fdeb1-26fa-4fc0-9e04-513c89601f40 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:14.320811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cb434-074e-4dbf-a6f5-f84d1aeef60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:14.321485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cb434-074e-4dbf-a6f5-f84d1aeef60b service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:14.321778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cb434-074e-4dbf-a6f5-f84d1aeef60b service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:14.322062 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cb434-074e-4dbf-a6f5-f84d1aeef60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:14.330848 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cb434-074e-4dbf-a6f5-f84d1aeef60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:14.332277 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cb434-074e-4dbf-a6f5-f84d1aeef60b service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:14.341579 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1429/2830] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:14.777650 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b52e1-fdf3-4981-93e9-9796ddd80a5b None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:14.779083 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-3b5b52e1-fdf3-4981-93e9-9796ddd80a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:14.793697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b52e1-fdf3-4981-93e9-9796ddd80a5b None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:14.798374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:14.799241 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:14.799526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:14.799806 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] PATH_INFO: `/v3/projects/7a8aca012120435788feb66ce0b090bc` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:14.801012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a8aca012120435788feb66ce0b090bc)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:14.802989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:14.917122 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a8aca012120435788feb66ce0b090bc', 'request_id': 'req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:15.042617 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a8aca012120435788feb66ce0b090bc', 'request_id': 'req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:15.048450 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a8aca012120435788feb66ce0b090bc', 'request_id': 'req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:15.059715 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] The token cache is being invalidate because project 7a8aca012120435788feb66ce0b090bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:09:15.060007 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:15.060640 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:15.062198 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1402/2831] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:09:14 2022] DELETE /identity/v3/projects/7a8aca012120435788feb66ce0b090bc => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:15.066146 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cb434-074e-4dbf-a6f5-f84d1aeef60b service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:15.076215 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cb434-074e-4dbf-a6f5-f84d1aeef60b service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:15.094222 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:15.130429 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f93561-a09f-439d-873c-ae156aa1abc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:15.131786 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f93561-a09f-439d-873c-ae156aa1abc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:15.132066 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f93561-a09f-439d-873c-ae156aa1abc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:15.132335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f93561-a09f-439d-873c-ae156aa1abc9 admin admin] PATH_INFO: `/v3/users/fa5cff3ad3514badb5a5bc1834fddda3` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:15.133770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f93561-a09f-439d-873c-ae156aa1abc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa5cff3ad3514badb5a5bc1834fddda3)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:15.136164 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f93561-a09f-439d-873c-ae156aa1abc9 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:15.332668 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f0f93561-a09f-439d-873c-ae156aa1abc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa5cff3ad3514badb5a5bc1834fddda3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:15.333342 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f0f93561-a09f-439d-873c-ae156aa1abc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa5cff3ad3514badb5a5bc1834fddda3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:15.340566 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f0f93561-a09f-439d-873c-ae156aa1abc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa5cff3ad3514badb5a5bc1834fddda3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:15.341186 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f0f93561-a09f-439d-873c-ae156aa1abc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa5cff3ad3514badb5a5bc1834fddda3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:15.354941 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-f0f93561-a09f-439d-873c-ae156aa1abc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa5cff3ad3514badb5a5bc1834fddda3'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:15.450893 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1430/2832] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:09:15 2022] DELETE /identity/v3/users/fa5cff3ad3514badb5a5bc1834fddda3 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:15.857509 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:15.859043 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:15.873732 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfde4a-09bc-4638-9e61-fc3f8be2a55b admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:15.920280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:15.931482 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:15.935155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:15.935155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:15.935155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] PATH_INFO: `/v3/projects/5ca47d311a784ec8bdc3df26d8c984eb` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:15.935155 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ca47d311a784ec8bdc3df26d8c984eb)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:15.935983 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:16.041412 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ca47d311a784ec8bdc3df26d8c984eb', 'request_id': 'req-fb93606f-0bb0-48cb-8f7a-1f016aec402d'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:16.151374 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ca47d311a784ec8bdc3df26d8c984eb', 'request_id': 'req-fb93606f-0bb0-48cb-8f7a-1f016aec402d'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:16.157221 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ca47d311a784ec8bdc3df26d8c984eb', 'request_id': 'req-fb93606f-0bb0-48cb-8f7a-1f016aec402d'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:16.168196 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] The token cache is being invalidate because project 5ca47d311a784ec8bdc3df26d8c984eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:09:16.168484 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:16.169110 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-fb93606f-0bb0-48cb-8f7a-1f016aec402d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:16.170903 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1403/2833] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:09:15 2022] DELETE /identity/v3/projects/5ca47d311a784ec8bdc3df26d8c984eb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:16.175570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3097a-b0cb-4d1c-8ee1-2c451ea43e33 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:16.176065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3097a-b0cb-4d1c-8ee1-2c451ea43e33 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:16.176179 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3097a-b0cb-4d1c-8ee1-2c451ea43e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:16.196183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f8f3097a-b0cb-4d1c-8ee1-2c451ea43e33 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:16.197476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f8f3097a-b0cb-4d1c-8ee1-2c451ea43e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:16.255315 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1431/2834] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:09:16.263778 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a47c73-f699-4809-985a-fe36a1d94500 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:16.264079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a47c73-f699-4809-985a-fe36a1d94500 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:16.264352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a47c73-f699-4809-985a-fe36a1d94500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:16.282160 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-11a47c73-f699-4809-985a-fe36a1d94500 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:16.283363 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-11a47c73-f699-4809-985a-fe36a1d94500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:16.291175 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1404/2835] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:16.295192 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3097a-b0cb-4d1c-8ee1-2c451ea43e33 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:16.305155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3097a-b0cb-4d1c-8ee1-2c451ea43e33 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:16.310177 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a13738-4891-4970-b9fd-ebc7b6932730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:16.310822 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a13738-4891-4970-b9fd-ebc7b6932730 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:16.311104 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a13738-4891-4970-b9fd-ebc7b6932730 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:16.311383 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a13738-4891-4970-b9fd-ebc7b6932730 admin admin] PATH_INFO: `/v3/domains` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:16.312098 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a13738-4891-4970-b9fd-ebc7b6932730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92270) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:09:16.312406 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a13738-4891-4970-b9fd-ebc7b6932730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:16.314054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a13738-4891-4970-b9fd-ebc7b6932730 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:16.326126 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1432/2836] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:09:16.330224 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a47c73-f699-4809-985a-fe36a1d94500 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:16.339342 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a47c73-f699-4809-985a-fe36a1d94500 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:16.344038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca6f8-fd39-40a6-93cb-ac92fb983c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:16.344687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca6f8-fd39-40a6-93cb-ac92fb983c85 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:16.344978 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca6f8-fd39-40a6-93cb-ac92fb983c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:16.345280 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca6f8-fd39-40a6-93cb-ac92fb983c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:16.345990 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aca6f8-fd39-40a6-93cb-ac92fb983c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:16.347927 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aca6f8-fd39-40a6-93cb-ac92fb983c85 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:16.469095 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1405/2837] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:09:16 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:09:16.473147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a13738-4891-4970-b9fd-ebc7b6932730 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:16.483046 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a13738-4891-4970-b9fd-ebc7b6932730 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:16.510009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:16.539706 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604d4cd-b649-44a7-ad3c-ce2e1059572e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:16.540363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604d4cd-b649-44a7-ad3c-ce2e1059572e admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:16.540645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604d4cd-b649-44a7-ad3c-ce2e1059572e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:16.540921 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604d4cd-b649-44a7-ad3c-ce2e1059572e admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:16.541634 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604d4cd-b649-44a7-ad3c-ce2e1059572e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:16.543523 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604d4cd-b649-44a7-ad3c-ce2e1059572e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:16.699911 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1433/2838] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:09:16 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:09:16.704011 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca6f8-fd39-40a6-93cb-ac92fb983c85 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:16.714045 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca6f8-fd39-40a6-93cb-ac92fb983c85 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:16.718497 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27edac5-a20b-4ad6-b7aa-fe15d59583f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:16.719472 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27edac5-a20b-4ad6-b7aa-fe15d59583f1 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:16.719713 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27edac5-a20b-4ad6-b7aa-fe15d59583f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:16.719986 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27edac5-a20b-4ad6-b7aa-fe15d59583f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:16.720693 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27edac5-a20b-4ad6-b7aa-fe15d59583f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:16.722321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27edac5-a20b-4ad6-b7aa-fe15d59583f1 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:16.732941 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1406/2839] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:16.740075 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604d4cd-b649-44a7-ad3c-ce2e1059572e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:16.750841 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604d4cd-b649-44a7-ad3c-ce2e1059572e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:16.755498 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b9be3-d513-4592-a0ce-6d68694780a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:16.756151 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b9be3-d513-4592-a0ce-6d68694780a0 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:16.756438 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b9be3-d513-4592-a0ce-6d68694780a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:16.756718 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b9be3-d513-4592-a0ce-6d68694780a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:16.757469 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b9be3-d513-4592-a0ce-6d68694780a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:16.759049 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b9be3-d513-4592-a0ce-6d68694780a0 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:16.768785 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1434/2840] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:16.772827 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27edac5-a20b-4ad6-b7aa-fe15d59583f1 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:16.782539 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27edac5-a20b-4ad6-b7aa-fe15d59583f1 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:16.787073 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460b334-2201-4741-957a-1034fae381a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:16.788452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460b334-2201-4741-957a-1034fae381a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:16.788733 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460b334-2201-4741-957a-1034fae381a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:16.789008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460b334-2201-4741-957a-1034fae381a0 admin admin] PATH_INFO: `/v3/projects/3959f49520054e6e938ff92c0fa07c84/users/6cba9665f1a14a9da756d11fb8cf9c0f/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:16.803012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460b334-2201-4741-957a-1034fae381a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3959f49520054e6e938ff92c0fa07c84, user_id=6cba9665f1a14a9da756d11fb8cf9c0f, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:16.806228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460b334-2201-4741-957a-1034fae381a0 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:16.919279 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1407/2841] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:09:16 2022] PUT /identity/v3/projects/3959f49520054e6e938ff92c0fa07c84/users/6cba9665f1a14a9da756d11fb8cf9c0f/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:16.924433 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006c810-dcb7-48d5-b15f-bd85a0b7b31b None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:16.924730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006c810-dcb7-48d5-b15f-bd85a0b7b31b None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:16.925039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006c810-dcb7-48d5-b15f-bd85a0b7b31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:16.942797 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f006c810-dcb7-48d5-b15f-bd85a0b7b31b None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:16.944622 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f006c810-dcb7-48d5-b15f-bd85a0b7b31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:17.003716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1435/2842] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:17.014396 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460b334-2201-4741-957a-1034fae381a0 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:17.016521 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-f460b334-2201-4741-957a-1034fae381a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:17.072480 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460b334-2201-4741-957a-1034fae381a0 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:17.076228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0ecea-e56c-4b7d-b7bd-bbdd2a705128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:17.076860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0ecea-e56c-4b7d-b7bd-bbdd2a705128 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:17.077147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0ecea-e56c-4b7d-b7bd-bbdd2a705128 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:17.077471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0ecea-e56c-4b7d-b7bd-bbdd2a705128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:17.086501 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0ecea-e56c-4b7d-b7bd-bbdd2a705128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:17.087967 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0ecea-e56c-4b7d-b7bd-bbdd2a705128 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:17.097603 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1408/2843] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:18.382993 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f141064-9858-40b3-ac74-521946992ef5 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:18.383541 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f141064-9858-40b3-ac74-521946992ef5 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:18.383647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f141064-9858-40b3-ac74-521946992ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:18.411199 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-9f141064-9858-40b3-ac74-521946992ef5 None None] MFA Rules not processed for user `6cba9665f1a14a9da756d11fb8cf9c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:18.412479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9f141064-9858-40b3-ac74-521946992ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:18.478107 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1436/2844] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:18.482711 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0ecea-e56c-4b7d-b7bd-bbdd2a705128 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:18.493108 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0ecea-e56c-4b7d-b7bd-bbdd2a705128 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:18.497995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089de2b4-5fb8-4140-99a2-27f76e2683ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:18.498646 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089de2b4-5fb8-4140-99a2-27f76e2683ff admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:18.498932 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089de2b4-5fb8-4140-99a2-27f76e2683ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:18.499212 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089de2b4-5fb8-4140-99a2-27f76e2683ff admin admin] PATH_INFO: `/v3/projects` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:18.500019 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089de2b4-5fb8-4140-99a2-27f76e2683ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:18.501999 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089de2b4-5fb8-4140-99a2-27f76e2683ff admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:18.517760 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:18.660908 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1409/2845] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:09:18 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:09:18.665090 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f141064-9858-40b3-ac74-521946992ef5 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:18.675775 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f141064-9858-40b3-ac74-521946992ef5 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:18.709028 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:18.731002 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db33777-7df2-48ca-9530-3c2754f7396b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:18.731688 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33777-7df2-48ca-9530-3c2754f7396b admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:18.731972 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33777-7df2-48ca-9530-3c2754f7396b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:18.732247 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db33777-7df2-48ca-9530-3c2754f7396b admin admin] PATH_INFO: `/v3/users` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:18.732925 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db33777-7df2-48ca-9530-3c2754f7396b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:18.734850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db33777-7df2-48ca-9530-3c2754f7396b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:18.850805 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1437/2846] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:09:18 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:09:18.854735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089de2b4-5fb8-4140-99a2-27f76e2683ff admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:18.865376 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089de2b4-5fb8-4140-99a2-27f76e2683ff admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:18.870441 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eba56-8f1c-4b8d-adaa-cad9058a9dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:18.871344 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eba56-8f1c-4b8d-adaa-cad9058a9dac admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:18.871719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eba56-8f1c-4b8d-adaa-cad9058a9dac admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:18.871997 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eba56-8f1c-4b8d-adaa-cad9058a9dac admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:18.872704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268eba56-8f1c-4b8d-adaa-cad9058a9dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:18.874327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268eba56-8f1c-4b8d-adaa-cad9058a9dac admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:18.885293 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1410/2847] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:18.889388 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db33777-7df2-48ca-9530-3c2754f7396b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:18.899811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db33777-7df2-48ca-9530-3c2754f7396b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:18.904867 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d77210-6d29-460e-afec-b45c0d5d259d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:18.905790 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d77210-6d29-460e-afec-b45c0d5d259d admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:18.906105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d77210-6d29-460e-afec-b45c0d5d259d admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:18.906382 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d77210-6d29-460e-afec-b45c0d5d259d admin admin] PATH_INFO: `/v3/projects/3836fbd8c9244394af6689a283ea9d0d/users/c7fbebeca13b46948930e4251f088d0f/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:18.920759 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d77210-6d29-460e-afec-b45c0d5d259d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3836fbd8c9244394af6689a283ea9d0d, user_id=c7fbebeca13b46948930e4251f088d0f, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:18.923372 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d77210-6d29-460e-afec-b45c0d5d259d admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:18.987275 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1438/2848] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:09:18 2022] PUT /identity/v3/projects/3836fbd8c9244394af6689a283ea9d0d/users/c7fbebeca13b46948930e4251f088d0f/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:18.991264 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eba56-8f1c-4b8d-adaa-cad9058a9dac admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:19.001638 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eba56-8f1c-4b8d-adaa-cad9058a9dac admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:19.056765 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc43b876-7bd9-48d5-b588-705fa8638611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:19.057475 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43b876-7bd9-48d5-b588-705fa8638611 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:19.057762 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43b876-7bd9-48d5-b588-705fa8638611 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:19.058038 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc43b876-7bd9-48d5-b588-705fa8638611 admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:19.058749 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43b876-7bd9-48d5-b588-705fa8638611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:19.060372 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc43b876-7bd9-48d5-b588-705fa8638611 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:19.070452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1411/2849] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:19.074555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d77210-6d29-460e-afec-b45c0d5d259d admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:19.085152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d77210-6d29-460e-afec-b45c0d5d259d admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:19.089748 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8572aa-c74b-4d12-a547-8f113f76d0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:19.090601 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8572aa-c74b-4d12-a547-8f113f76d0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:19.090881 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8572aa-c74b-4d12-a547-8f113f76d0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:19.091150 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8572aa-c74b-4d12-a547-8f113f76d0ee admin admin] PATH_INFO: `/v3/projects/3836fbd8c9244394af6689a283ea9d0d/users/c7fbebeca13b46948930e4251f088d0f/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:19.093086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8572aa-c74b-4d12-a547-8f113f76d0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=3836fbd8c9244394af6689a283ea9d0d, user_id=c7fbebeca13b46948930e4251f088d0f, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:19.095703 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8572aa-c74b-4d12-a547-8f113f76d0ee admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:19.105695 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1439/2850] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:09:19 2022] PUT /identity/v3/projects/3836fbd8c9244394af6689a283ea9d0d/users/c7fbebeca13b46948930e4251f088d0f/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:20.418729 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14a338-ae60-48f5-a81d-6a270747eaca None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:20.419323 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14a338-ae60-48f5-a81d-6a270747eaca None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:20.419444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14a338-ae60-48f5-a81d-6a270747eaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:20.446537 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-9b14a338-ae60-48f5-a81d-6a270747eaca None None] MFA Rules not processed for user `c7fbebeca13b46948930e4251f088d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:20.447782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-9b14a338-ae60-48f5-a81d-6a270747eaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:20.517173 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1412/2851] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:20 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:20.528889 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8572aa-c74b-4d12-a547-8f113f76d0ee admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:20.530573 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-5c8572aa-c74b-4d12-a547-8f113f76d0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:20.587301 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8572aa-c74b-4d12-a547-8f113f76d0ee admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:20.590896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf4518-9d70-43c2-8489-066cd1a3b8b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:20.591527 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf4518-9d70-43c2-8489-066cd1a3b8b1 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:20.591811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf4518-9d70-43c2-8489-066cd1a3b8b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:20.592114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf4518-9d70-43c2-8489-066cd1a3b8b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:20.647937 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf4518-9d70-43c2-8489-066cd1a3b8b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:20.649396 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf4518-9d70-43c2-8489-066cd1a3b8b1 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:20.658271 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1440/2852] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:22.339790 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14a338-ae60-48f5-a81d-6a270747eaca None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:22.341197 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-9b14a338-ae60-48f5-a81d-6a270747eaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:22.353041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14a338-ae60-48f5-a81d-6a270747eaca None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:22.356556 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f62a3-6ca5-4c95-9f46-55915cab8191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:22.357195 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f62a3-6ca5-4c95-9f46-55915cab8191 service cinder] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:22.357515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f62a3-6ca5-4c95-9f46-55915cab8191 service cinder] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:22.357800 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f62a3-6ca5-4c95-9f46-55915cab8191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:22.366860 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f62a3-6ca5-4c95-9f46-55915cab8191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:22.368792 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f62a3-6ca5-4c95-9f46-55915cab8191 service cinder] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:22.378488 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1413/2853] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:22 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:30.654041 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:32.374249 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=76 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:37.339746 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf4518-9d70-43c2-8489-066cd1a3b8b1 service cinder] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:37.341216 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-d7bf4518-9d70-43c2-8489-066cd1a3b8b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:37.413690 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf4518-9d70-43c2-8489-066cd1a3b8b1 service cinder] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:37.419206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216781b-82e7-4c6e-94a4-1a1b7e91ed10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:37.419844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216781b-82e7-4c6e-94a4-1a1b7e91ed10 service placement] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:37.420160 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216781b-82e7-4c6e-94a4-1a1b7e91ed10 service placement] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:37.420440 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216781b-82e7-4c6e-94a4-1a1b7e91ed10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:37.422208 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-9216781b-82e7-4c6e-94a4-1a1b7e91ed10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:37.433339 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216781b-82e7-4c6e-94a4-1a1b7e91ed10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:37.436095 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216781b-82e7-4c6e-94a4-1a1b7e91ed10 service placement] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:37.444485 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1441/2854] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:38.347828 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f62a3-6ca5-4c95-9f46-55915cab8191 service cinder] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:38.413230 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f62a3-6ca5-4c95-9f46-55915cab8191 service cinder] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:38.418266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:38.419131 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:38.419418 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:38.419698 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] PATH_INFO: `/v3/users/6cba9665f1a14a9da756d11fb8cf9c0f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:38.420859 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cba9665f1a14a9da756d11fb8cf9c0f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:38.422801 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:38.588879 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cba9665f1a14a9da756d11fb8cf9c0f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:38.595596 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cba9665f1a14a9da756d11fb8cf9c0f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:38.596262 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cba9665f1a14a9da756d11fb8cf9c0f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:38.608445 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cba9665f1a14a9da756d11fb8cf9c0f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:38.609082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cba9665f1a14a9da756d11fb8cf9c0f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:38.695404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1414/2855] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:09:38 2022] DELETE /identity/v3/users/6cba9665f1a14a9da756d11fb8cf9c0f => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:38.966552 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216781b-82e7-4c6e-94a4-1a1b7e91ed10 service placement] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:38.977489 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216781b-82e7-4c6e-94a4-1a1b7e91ed10 service placement] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:39.035204 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:39.036054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:39.036361 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:39.036632 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] PATH_INFO: `/v3/projects/3959f49520054e6e938ff92c0fa07c84` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:39.037811 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3959f49520054e6e938ff92c0fa07c84)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:39.039824 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:39.145114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3959f49520054e6e938ff92c0fa07c84', 'request_id': 'req-c5a695da-d774-4238-a21b-3c061677216e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:39.244680 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3959f49520054e6e938ff92c0fa07c84', 'request_id': 'req-c5a695da-d774-4238-a21b-3c061677216e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:39.250976 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3959f49520054e6e938ff92c0fa07c84', 'request_id': 'req-c5a695da-d774-4238-a21b-3c061677216e'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:39.262869 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] The token cache is being invalidate because project 3959f49520054e6e938ff92c0fa07c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:09:39.263155 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:39.263807 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:39.265329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1442/2856] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:09:38 2022] DELETE /identity/v3/projects/3959f49520054e6e938ff92c0fa07c84 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:39.269192 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:39.270601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:39.283632 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ae70c-3507-4834-9ce1-3434ee46917c admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:39.337839 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242f839-16fd-40f3-a1d2-839416952770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:39.339307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f839-16fd-40f3-a1d2-839416952770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:39.339597 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f839-16fd-40f3-a1d2-839416952770 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:39.339871 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242f839-16fd-40f3-a1d2-839416952770 admin admin] PATH_INFO: `/v3/users/c7fbebeca13b46948930e4251f088d0f` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:39.341041 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242f839-16fd-40f3-a1d2-839416952770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7fbebeca13b46948930e4251f088d0f)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:39.343018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242f839-16fd-40f3-a1d2-839416952770 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:39.363920 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:39.566169 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7242f839-16fd-40f3-a1d2-839416952770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7fbebeca13b46948930e4251f088d0f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:39.574132 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7242f839-16fd-40f3-a1d2-839416952770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7fbebeca13b46948930e4251f088d0f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:39.575858 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7242f839-16fd-40f3-a1d2-839416952770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7fbebeca13b46948930e4251f088d0f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:39.586454 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7242f839-16fd-40f3-a1d2-839416952770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7fbebeca13b46948930e4251f088d0f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:39.587135 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-7242f839-16fd-40f3-a1d2-839416952770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7fbebeca13b46948930e4251f088d0f'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:39.666664 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1415/2857] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:09:39 2022] DELETE /identity/v3/users/c7fbebeca13b46948930e4251f088d0f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:39.932181 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:39.934688 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:39.948227 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a695da-d774-4238-a21b-3c061677216e admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:39.966190 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:40.009778 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895829df-82da-42bf-a737-def55e96431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:40.010656 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895829df-82da-42bf-a737-def55e96431f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:40.010939 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895829df-82da-42bf-a737-def55e96431f admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:40.011213 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895829df-82da-42bf-a737-def55e96431f admin admin] PATH_INFO: `/v3/projects/3836fbd8c9244394af6689a283ea9d0d` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:40.012412 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895829df-82da-42bf-a737-def55e96431f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3836fbd8c9244394af6689a283ea9d0d)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:40.014482 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895829df-82da-42bf-a737-def55e96431f admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:40.124556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-895829df-82da-42bf-a737-def55e96431f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3836fbd8c9244394af6689a283ea9d0d', 'request_id': 'req-895829df-82da-42bf-a737-def55e96431f'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:40.181657 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-895829df-82da-42bf-a737-def55e96431f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3836fbd8c9244394af6689a283ea9d0d', 'request_id': 'req-895829df-82da-42bf-a737-def55e96431f'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:40.187329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-895829df-82da-42bf-a737-def55e96431f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3836fbd8c9244394af6689a283ea9d0d', 'request_id': 'req-895829df-82da-42bf-a737-def55e96431f'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:40.198378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-895829df-82da-42bf-a737-def55e96431f admin admin] The token cache is being invalidate because project 3836fbd8c9244394af6689a283ea9d0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:09:40.198672 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-895829df-82da-42bf-a737-def55e96431f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:40.199335 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-895829df-82da-42bf-a737-def55e96431f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:40.200876 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1443/2858] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:09:39 2022] DELETE /identity/v3/projects/3836fbd8c9244394af6689a283ea9d0d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:40.205982 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931e37-637c-488b-b957-ae29967ee07a None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:40.206287 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931e37-637c-488b-b957-ae29967ee07a None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:40.206599 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14931e37-637c-488b-b957-ae29967ee07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:40.225157 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-14931e37-637c-488b-b957-ae29967ee07a None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:40.226409 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-14931e37-637c-488b-b957-ae29967ee07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:40.285051 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1416/2859] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:40.293428 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da92f8c-bde0-472b-8c9c-3a984ab12088 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:40.293716 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da92f8c-bde0-472b-8c9c-3a984ab12088 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:40.293993 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da92f8c-bde0-472b-8c9c-3a984ab12088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:40.312360 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4da92f8c-bde0-472b-8c9c-3a984ab12088 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:40.313662 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4da92f8c-bde0-472b-8c9c-3a984ab12088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:40.321670 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1444/2860] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:40.325811 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14931e37-637c-488b-b957-ae29967ee07a None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:40.336018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14931e37-637c-488b-b957-ae29967ee07a None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:40.341425 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9230e0-4c98-495c-bcf2-76fa5e3b2f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:40.342103 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9230e0-4c98-495c-bcf2-76fa5e3b2f21 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:40.342404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9230e0-4c98-495c-bcf2-76fa5e3b2f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:40.342663 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9230e0-4c98-495c-bcf2-76fa5e3b2f21 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:40.343398 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9230e0-4c98-495c-bcf2-76fa5e3b2f21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:09:40.343698 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9230e0-4c98-495c-bcf2-76fa5e3b2f21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:40.345334 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9230e0-4c98-495c-bcf2-76fa5e3b2f21 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:40.358365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1417/2861] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:09:40.362813 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da92f8c-bde0-472b-8c9c-3a984ab12088 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:40.372379 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da92f8c-bde0-472b-8c9c-3a984ab12088 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:40.376959 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129ebc8-f982-4f29-a7fa-946385ab4aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:40.377628 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129ebc8-f982-4f29-a7fa-946385ab4aeb admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:40.377910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129ebc8-f982-4f29-a7fa-946385ab4aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:40.378183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6129ebc8-f982-4f29-a7fa-946385ab4aeb admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:40.378868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129ebc8-f982-4f29-a7fa-946385ab4aeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:40.380732 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6129ebc8-f982-4f29-a7fa-946385ab4aeb admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:40.456730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1445/2862] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:09:40 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:09:40.460822 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9230e0-4c98-495c-bcf2-76fa5e3b2f21 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:40.471405 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9230e0-4c98-495c-bcf2-76fa5e3b2f21 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:40.526364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99e5ac-bbc7-44b0-b329-3c3bf0defa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:40.527050 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99e5ac-bbc7-44b0-b329-3c3bf0defa22 admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:40.527321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99e5ac-bbc7-44b0-b329-3c3bf0defa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:40.527601 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab99e5ac-bbc7-44b0-b329-3c3bf0defa22 admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:40.528291 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99e5ac-bbc7-44b0-b329-3c3bf0defa22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:40.530240 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab99e5ac-bbc7-44b0-b329-3c3bf0defa22 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:40.654077 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1418/2863] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:09:40 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:09:40.658237 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129ebc8-f982-4f29-a7fa-946385ab4aeb admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:40.668517 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6129ebc8-f982-4f29-a7fa-946385ab4aeb admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:40.673067 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c59560-d6de-4ea9-94ef-431eb40ed0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:40.673751 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c59560-d6de-4ea9-94ef-431eb40ed0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:40.674039 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c59560-d6de-4ea9-94ef-431eb40ed0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:40.674318 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c59560-d6de-4ea9-94ef-431eb40ed0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:40.675065 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c59560-d6de-4ea9-94ef-431eb40ed0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:40.676645 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c59560-d6de-4ea9-94ef-431eb40ed0f5 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:40.686966 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1446/2864] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:40.690788 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99e5ac-bbc7-44b0-b329-3c3bf0defa22 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:40.701130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab99e5ac-bbc7-44b0-b329-3c3bf0defa22 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:40.705686 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440af161-2b44-415e-bcff-d92e4864fa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:40.706329 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440af161-2b44-415e-bcff-d92e4864fa0f admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:40.706611 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440af161-2b44-415e-bcff-d92e4864fa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:40.706933 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440af161-2b44-415e-bcff-d92e4864fa0f admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:40.707591 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440af161-2b44-415e-bcff-d92e4864fa0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:40.709161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440af161-2b44-415e-bcff-d92e4864fa0f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:40.719319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1419/2865] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:40.723162 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c59560-d6de-4ea9-94ef-431eb40ed0f5 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:40.732910 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c59560-d6de-4ea9-94ef-431eb40ed0f5 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:40.737542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e38877-0b14-4377-ab10-6124d1f005d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:40.738422 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e38877-0b14-4377-ab10-6124d1f005d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:40.738696 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e38877-0b14-4377-ab10-6124d1f005d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:40.738980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e38877-0b14-4377-ab10-6124d1f005d3 admin admin] PATH_INFO: `/v3/projects/a5b339d0207d485db6be869c9b4a2222/users/ed8ef2ba79ad457c8ea5f0dc4dba5f00/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:40.753333 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e38877-0b14-4377-ab10-6124d1f005d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b339d0207d485db6be869c9b4a2222, user_id=ed8ef2ba79ad457c8ea5f0dc4dba5f00, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:40.755994 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e38877-0b14-4377-ab10-6124d1f005d3 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:40.859211 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1447/2866] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:09:40 2022] PUT /identity/v3/projects/a5b339d0207d485db6be869c9b4a2222/users/ed8ef2ba79ad457c8ea5f0dc4dba5f00/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:40.870040 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa48fe-1ebf-463a-a6aa-445b11f38d64 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:40.870471 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa48fe-1ebf-463a-a6aa-445b11f38d64 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:40.870748 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa48fe-1ebf-463a-a6aa-445b11f38d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:40.877825 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:40.897319 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-b6fa48fe-1ebf-463a-a6aa-445b11f38d64 None None] MFA Rules not processed for user `ed8ef2ba79ad457c8ea5f0dc4dba5f00`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:40.904572 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b6fa48fe-1ebf-463a-a6aa-445b11f38d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:40.960549 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1420/2867] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:09:40.971077 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e38877-0b14-4377-ab10-6124d1f005d3 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:40.975378 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-40e38877-0b14-4377-ab10-6124d1f005d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:41.024476 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:41.029568 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e38877-0b14-4377-ab10-6124d1f005d3 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:41.033120 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857811c-ceca-4c8c-aa27-9601c6ddf971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c82335b7ec74fb3a1508a2fc76335ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:41.033827 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857811c-ceca-4c8c-aa27-9601c6ddf971 service cinder] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:41.034113 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857811c-ceca-4c8c-aa27-9601c6ddf971 service cinder] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:41.034390 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857811c-ceca-4c8c-aa27-9601c6ddf971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:41.042638 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857811c-ceca-4c8c-aa27-9601c6ddf971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:41.044115 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857811c-ceca-4c8c-aa27-9601c6ddf971 service cinder] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:41.052702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1448/2868] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:50.992726 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=30 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:51.136016 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=5 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:51.391341 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa48fe-1ebf-463a-a6aa-445b11f38d64 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:51.450874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa48fe-1ebf-463a-a6aa-445b11f38d64 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:51.455259 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:51.456121 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:51.456400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:51.456671 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] PATH_INFO: `/v3/users/ed8ef2ba79ad457c8ea5f0dc4dba5f00` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:51.457845 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed8ef2ba79ad457c8ea5f0dc4dba5f00)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:51.459746 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:51.703266 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed8ef2ba79ad457c8ea5f0dc4dba5f00'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:51.709861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed8ef2ba79ad457c8ea5f0dc4dba5f00'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:51.710517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed8ef2ba79ad457c8ea5f0dc4dba5f00'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:51.722582 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed8ef2ba79ad457c8ea5f0dc4dba5f00'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:51.723220 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed8ef2ba79ad457c8ea5f0dc4dba5f00'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:51.813264 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1421/2869] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:09:51 2022] DELETE /identity/v3/users/ed8ef2ba79ad457c8ea5f0dc4dba5f00 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:51.817919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7171c12-2f48-4a9f-ac22-722aa261c1b1 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:51.818353 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7171c12-2f48-4a9f-ac22-722aa261c1b1 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:51.818738 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7171c12-2f48-4a9f-ac22-722aa261c1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:51.836185 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f7171c12-2f48-4a9f-ac22-722aa261c1b1 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:51.837431 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f7171c12-2f48-4a9f-ac22-722aa261c1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:51.899723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1449/2870] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:51.909977 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:51.911411 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:51.969178 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0ccd4-3c4c-4a64-859b-54bff99c841d admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:51.972789 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba5526-a282-407f-a657-0076a513566d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:51.973463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba5526-a282-407f-a657-0076a513566d service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:51.973750 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba5526-a282-407f-a657-0076a513566d service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:51.974024 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba5526-a282-407f-a657-0076a513566d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:51.983515 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba5526-a282-407f-a657-0076a513566d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:51.984945 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba5526-a282-407f-a657-0076a513566d service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:51.994253 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1422/2871] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:52.398282 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7171c12-2f48-4a9f-ac22-722aa261c1b1 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:52.399739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f7171c12-2f48-4a9f-ac22-722aa261c1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:52.412549 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7171c12-2f48-4a9f-ac22-722aa261c1b1 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:52.417343 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:52.421397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:52.421397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:52.421397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] PATH_INFO: `/v3/projects/a5b339d0207d485db6be869c9b4a2222` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:52.421397 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b339d0207d485db6be869c9b4a2222)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:52.422540 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:52.530058 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b339d0207d485db6be869c9b4a2222', 'request_id': 'req-37543c1f-884c-45f1-b747-4c2de66956f6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:52.621550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b339d0207d485db6be869c9b4a2222', 'request_id': 'req-37543c1f-884c-45f1-b747-4c2de66956f6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:52.627501 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b339d0207d485db6be869c9b4a2222', 'request_id': 'req-37543c1f-884c-45f1-b747-4c2de66956f6'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:52.638643 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] The token cache is being invalidate because project a5b339d0207d485db6be869c9b4a2222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92270) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:09:52.638936 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:52.639576 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-37543c1f-884c-45f1-b747-4c2de66956f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:09:52.641105 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1450/2872] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:09:52 2022] DELETE /identity/v3/projects/a5b339d0207d485db6be869c9b4a2222 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:52.646014 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f614343-7592-44f4-9204-f0e98f7b4e65 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:52.646404 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f614343-7592-44f4-9204-f0e98f7b4e65 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:52.646687 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f614343-7592-44f4-9204-f0e98f7b4e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:52.664610 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6f614343-7592-44f4-9204-f0e98f7b4e65 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:52.666010 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6f614343-7592-44f4-9204-f0e98f7b4e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:52.683662 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:52.724510 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1423/2873] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:52.733561 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede66b9-3b80-41b7-a32b-70335d6b390c None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:52.733860 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede66b9-3b80-41b7-a32b-70335d6b390c None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:52.734154 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede66b9-3b80-41b7-a32b-70335d6b390c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:52.752702 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4ede66b9-3b80-41b7-a32b-70335d6b390c None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:52.753957 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4ede66b9-3b80-41b7-a32b-70335d6b390c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:52.762467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1451/2874] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:52.766548 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f614343-7592-44f4-9204-f0e98f7b4e65 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:52.775883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f614343-7592-44f4-9204-f0e98f7b4e65 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:52.781105 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc2972-1f9c-47c2-a986-bf1f82e89982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:52.781727 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc2972-1f9c-47c2-a986-bf1f82e89982 admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:52.782015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc2972-1f9c-47c2-a986-bf1f82e89982 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:52.782298 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc2972-1f9c-47c2-a986-bf1f82e89982 admin admin] PATH_INFO: `/v3/domains` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:52.783008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc2972-1f9c-47c2-a986-bf1f82e89982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 19:09:52.783321 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc2972-1f9c-47c2-a986-bf1f82e89982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:52.784926 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc2972-1f9c-47c2-a986-bf1f82e89982 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:52.796018 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1424/2875] 10.222.0.81 () {58 vars in 1152 bytes} [Thu Jun 16 19:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 19:09:52.800182 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede66b9-3b80-41b7-a32b-70335d6b390c None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:52.809294 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede66b9-3b80-41b7-a32b-70335d6b390c None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:52.814054 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef48d71-6add-4937-a1d8-43bec19ca964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:52.814723 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef48d71-6add-4937-a1d8-43bec19ca964 admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:52.815009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef48d71-6add-4937-a1d8-43bec19ca964 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:52.815284 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef48d71-6add-4937-a1d8-43bec19ca964 admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:52.815967 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef48d71-6add-4937-a1d8-43bec19ca964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:52.817919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef48d71-6add-4937-a1d8-43bec19ca964 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:52.896808 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1452/2876] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:09:52 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:09:52.900819 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc2972-1f9c-47c2-a986-bf1f82e89982 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:52.911463 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc2972-1f9c-47c2-a986-bf1f82e89982 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:52.967874 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183be7f-db6a-4d7c-822e-8b8c5b3fd61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:52.968534 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183be7f-db6a-4d7c-822e-8b8c5b3fd61d admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:52.968834 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183be7f-db6a-4d7c-822e-8b8c5b3fd61d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:52.969091 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183be7f-db6a-4d7c-822e-8b8c5b3fd61d admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:52.969862 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183be7f-db6a-4d7c-822e-8b8c5b3fd61d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:52.971781 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183be7f-db6a-4d7c-822e-8b8c5b3fd61d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:53.079060 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1425/2877] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:09:52 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:09:53.083256 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef48d71-6add-4937-a1d8-43bec19ca964 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:53.094555 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef48d71-6add-4937-a1d8-43bec19ca964 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:53.099060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8444f-f9fc-4332-b21e-4f57fd2cab9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:53.099707 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8444f-f9fc-4332-b21e-4f57fd2cab9c admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:53.099989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8444f-f9fc-4332-b21e-4f57fd2cab9c admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:53.100398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8444f-f9fc-4332-b21e-4f57fd2cab9c admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:53.101119 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8444f-f9fc-4332-b21e-4f57fd2cab9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:53.102734 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8444f-f9fc-4332-b21e-4f57fd2cab9c admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:53.113435 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1453/2878] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:53.117473 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183be7f-db6a-4d7c-822e-8b8c5b3fd61d admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:53.128347 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183be7f-db6a-4d7c-822e-8b8c5b3fd61d admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:53.132901 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62adc3-bbf0-44c2-9398-dd4984b7074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:53.133596 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62adc3-bbf0-44c2-9398-dd4984b7074d admin admin] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:53.133885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62adc3-bbf0-44c2-9398-dd4984b7074d admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:53.134163 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62adc3-bbf0-44c2-9398-dd4984b7074d admin admin] PATH_INFO: `/v3/roles` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:53.134885 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62adc3-bbf0-44c2-9398-dd4984b7074d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:53.136470 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62adc3-bbf0-44c2-9398-dd4984b7074d admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:53.146964 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1426/2879] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:53.151114 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8444f-f9fc-4332-b21e-4f57fd2cab9c admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:53.161112 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8444f-f9fc-4332-b21e-4f57fd2cab9c admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:53.165833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67d2d2-837d-48a2-bd42-889e0bb578c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:53.166703 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67d2d2-837d-48a2-bd42-889e0bb578c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:53.166989 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67d2d2-837d-48a2-bd42-889e0bb578c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:53.167270 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67d2d2-837d-48a2-bd42-889e0bb578c8 admin admin] PATH_INFO: `/v3/projects/434965eee025410ca595f90a8bd80ee5/users/a52eccb2ea4b4920863f4e74e75296ab/roles/0960434fd9da404ea96841eb4306eaae` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:53.181711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67d2d2-837d-48a2-bd42-889e0bb578c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=434965eee025410ca595f90a8bd80ee5, user_id=a52eccb2ea4b4920863f4e74e75296ab, role_id=0960434fd9da404ea96841eb4306eaae)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:53.184868 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67d2d2-837d-48a2-bd42-889e0bb578c8 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:53.284101 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1454/2880] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:09:53 2022] PUT /identity/v3/projects/434965eee025410ca595f90a8bd80ee5/users/a52eccb2ea4b4920863f4e74e75296ab/roles/0960434fd9da404ea96841eb4306eaae => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:53.289307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692e5c7-1f44-4b6a-9c8b-53bf5b14c5d1 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:53.289676 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692e5c7-1f44-4b6a-9c8b-53bf5b14c5d1 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:53.289965 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6692e5c7-1f44-4b6a-9c8b-53bf5b14c5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:53.307456 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-6692e5c7-1f44-4b6a-9c8b-53bf5b14c5d1 None None] MFA Rules not processed for user `88fee257b59d48f2b418ffa9e86b6f8e`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:53.308706 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-6692e5c7-1f44-4b6a-9c8b-53bf5b14c5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:53.367098 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1427/2881] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 19:09:53.377389 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67d2d2-837d-48a2-bd42-889e0bb578c8 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:53.379085 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-7b67d2d2-837d-48a2-bd42-889e0bb578c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:53.400854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:53.435240 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67d2d2-837d-48a2-bd42-889e0bb578c8 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:53.438919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9d548-ace8-43c1-a112-c6ec8c0d8986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:53.439563 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d548-ace8-43c1-a112-c6ec8c0d8986 service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:53.439850 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d548-ace8-43c1-a112-c6ec8c0d8986 service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:53.440147 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d548-ace8-43c1-a112-c6ec8c0d8986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:53.449496 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9d548-ace8-43c1-a112-c6ec8c0d8986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:53.451479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb9d548-ace8-43c1-a112-c6ec8c0d8986 service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:53.461034 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1455/2882] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:09:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:09:59.006268 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab0e5c-fb6e-4bd6-b868-2cdc26609cb1 None None] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:59.006761 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab0e5c-fb6e-4bd6-b868-2cdc26609cb1 None None] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:59.006869 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab0e5c-fb6e-4bd6-b868-2cdc26609cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:59.035124 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.auth.core [None req-38ab0e5c-fb6e-4bd6-b868-2cdc26609cb1 None None] MFA Rules not processed for user `a52eccb2ea4b4920863f4e74e75296ab`. Rule list: `[]` (Enabled: `True`). {{(pid=92269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:09:59.036299 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-38ab0e5c-fb6e-4bd6-b868-2cdc26609cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:09:59.053824 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:09:59.104786 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1428/2883] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:09:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:09:59.109373 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9d548-ace8-43c1-a112-c6ec8c0d8986 service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:59.119183 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9d548-ace8-43c1-a112-c6ec8c0d8986 service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:59.124520 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14e1d1-e280-4edc-82b1-7aa4b023204b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:59.125173 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14e1d1-e280-4edc-82b1-7aa4b023204b admin admin] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:59.125508 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14e1d1-e280-4edc-82b1-7aa4b023204b admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:59.125788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14e1d1-e280-4edc-82b1-7aa4b023204b admin admin] PATH_INFO: `/v3/projects` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:59.126488 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14e1d1-e280-4edc-82b1-7aa4b023204b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:59.128539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14e1d1-e280-4edc-82b1-7aa4b023204b admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:59.232788 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1456/2884] 10.222.0.81 () {60 vars in 1152 bytes} [Thu Jun 16 19:09:59 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 19:09:59.236863 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab0e5c-fb6e-4bd6-b868-2cdc26609cb1 None None] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:59.248199 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab0e5c-fb6e-4bd6-b868-2cdc26609cb1 None None] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:59.304883 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937401a2-05dd-4480-987c-a29c64b0012f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:59.305589 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937401a2-05dd-4480-987c-a29c64b0012f admin admin] REQUEST_METHOD: `POST` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:59.305878 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937401a2-05dd-4480-987c-a29c64b0012f admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:59.306157 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937401a2-05dd-4480-987c-a29c64b0012f admin admin] PATH_INFO: `/v3/users` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:59.306850 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937401a2-05dd-4480-987c-a29c64b0012f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:59.309307 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937401a2-05dd-4480-987c-a29c64b0012f admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:59.414800 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1429/2885] 10.222.0.81 () {60 vars in 1143 bytes} [Thu Jun 16 19:09:59 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 19:09:59.418704 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14e1d1-e280-4edc-82b1-7aa4b023204b admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:59.428836 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14e1d1-e280-4edc-82b1-7aa4b023204b admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:59.433446 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450fbdc-be23-42fc-a3c1-d5e32e5e1e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:59.434595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450fbdc-be23-42fc-a3c1-d5e32e5e1e41 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:59.434875 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450fbdc-be23-42fc-a3c1-d5e32e5e1e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:59.435149 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450fbdc-be23-42fc-a3c1-d5e32e5e1e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:59.435896 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450fbdc-be23-42fc-a3c1-d5e32e5e1e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:59.437514 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450fbdc-be23-42fc-a3c1-d5e32e5e1e41 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:59.447821 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1457/2886] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:59.452400 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937401a2-05dd-4480-987c-a29c64b0012f admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:59.462315 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937401a2-05dd-4480-987c-a29c64b0012f admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:59.466939 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cb67d-0515-403b-85f4-40086864483a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:59.467814 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb67d-0515-403b-85f4-40086864483a admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:59.468099 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb67d-0515-403b-85f4-40086864483a admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:59.468371 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb67d-0515-403b-85f4-40086864483a admin admin] PATH_INFO: `/v3/projects/0fdb52841fa6484a9f3e0b8f5a1335b0/users/d95fc270ea7e4998b52870073396e366/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:59.482719 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cb67d-0515-403b-85f4-40086864483a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fdb52841fa6484a9f3e0b8f5a1335b0, user_id=d95fc270ea7e4998b52870073396e366, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:59.485364 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cb67d-0515-403b-85f4-40086864483a admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:59.594313 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1430/2887] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:09:59 2022] PUT /identity/v3/projects/0fdb52841fa6484a9f3e0b8f5a1335b0/users/d95fc270ea7e4998b52870073396e366/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:09:59.598289 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450fbdc-be23-42fc-a3c1-d5e32e5e1e41 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:59.609539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450fbdc-be23-42fc-a3c1-d5e32e5e1e41 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:59.665556 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43573ee2-fe59-4415-bafe-af260e1293b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:59.666152 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43573ee2-fe59-4415-bafe-af260e1293b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:59.666439 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43573ee2-fe59-4415-bafe-af260e1293b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:59.666743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43573ee2-fe59-4415-bafe-af260e1293b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:59.667467 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43573ee2-fe59-4415-bafe-af260e1293b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:59.669677 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43573ee2-fe59-4415-bafe-af260e1293b0 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:59.679844 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1458/2888] 10.222.0.81 () {58 vars in 1121 bytes} [Thu Jun 16 19:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 19:09:59.684524 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cb67d-0515-403b-85f4-40086864483a admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:09:59.694975 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cb67d-0515-403b-85f4-40086864483a admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:09:59.699724 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b4723-d8bf-4ee3-994f-c09270dcc597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:09:59.700629 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b4723-d8bf-4ee3-994f-c09270dcc597 admin admin] REQUEST_METHOD: `PUT` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:09:59.700914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b4723-d8bf-4ee3-994f-c09270dcc597 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:09:59.701191 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b4723-d8bf-4ee3-994f-c09270dcc597 admin admin] PATH_INFO: `/v3/projects/0fdb52841fa6484a9f3e0b8f5a1335b0/users/d95fc270ea7e4998b52870073396e366/roles/7bdd42ee67ad4b74a7d9b9738787ac99` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:09:59.703170 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b4723-d8bf-4ee3-994f-c09270dcc597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fdb52841fa6484a9f3e0b8f5a1335b0, user_id=d95fc270ea7e4998b52870073396e366, role_id=7bdd42ee67ad4b74a7d9b9738787ac99)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:09:59.705829 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b4723-d8bf-4ee3-994f-c09270dcc597 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:09:59.717127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1431/2889] 10.222.0.81 () {60 vars in 1482 bytes} [Thu Jun 16 19:09:59 2022] PUT /identity/v3/projects/0fdb52841fa6484a9f3e0b8f5a1335b0/users/d95fc270ea7e4998b52870073396e366/roles/7bdd42ee67ad4b74a7d9b9738787ac99 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:10:05.181349 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47c74c-e8c2-4069-958f-aaa259efde73 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:05.181946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47c74c-e8c2-4069-958f-aaa259efde73 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:05.181946 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47c74c-e8c2-4069-958f-aaa259efde73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:05.211770 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-cc47c74c-e8c2-4069-958f-aaa259efde73 None None] MFA Rules not processed for user `d95fc270ea7e4998b52870073396e366`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:10:05.213420 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-cc47c74c-e8c2-4069-958f-aaa259efde73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:05.287640 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1459/2890] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:10:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:10:05.300082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b4723-d8bf-4ee3-994f-c09270dcc597 admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:05.301947 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-3e1b4723-d8bf-4ee3-994f-c09270dcc597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:05.388200 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b4723-d8bf-4ee3-994f-c09270dcc597 admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:05.391877 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c3b8b-db6c-45e4-b2cd-fc2b3094218c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd17b7a7128d64b2ba548add02d3ec264', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:05.392525 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c3b8b-db6c-45e4-b2cd-fc2b3094218c service glance] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:05.392809 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c3b8b-db6c-45e4-b2cd-fc2b3094218c service glance] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:05.393087 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c3b8b-db6c-45e4-b2cd-fc2b3094218c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:05.397414 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:10:05.453146 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c3b8b-db6c-45e4-b2cd-fc2b3094218c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:05.454866 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c3b8b-db6c-45e4-b2cd-fc2b3094218c service glance] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:05.474579 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1432/2891] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:10:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:10:05.713109 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcf367-5c9b-4335-8c9d-c03b403bb4f7 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:05.713451 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcf367-5c9b-4335-8c9d-c03b403bb4f7 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:05.713743 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcf367-5c9b-4335-8c9d-c03b403bb4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:05.729326 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:10:05.773730 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-c2fcf367-5c9b-4335-8c9d-c03b403bb4f7 None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:10:05.774948 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-c2fcf367-5c9b-4335-8c9d-c03b403bb4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:05.843859 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1460/2892] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 19:10:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:10:05.992830 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c3b8b-db6c-45e4-b2cd-fc2b3094218c service glance] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:05.994426 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8b5c3b8b-db6c-45e4-b2cd-fc2b3094218c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:06.085020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c3b8b-db6c-45e4-b2cd-fc2b3094218c service glance] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:06.090526 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6f65d-b46f-448f-a226-7020a2fc39bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:06.091164 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6f65d-b46f-448f-a226-7020a2fc39bc service swift] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:06.091443 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6f65d-b46f-448f-a226-7020a2fc39bc service swift] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:06.091714 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6f65d-b46f-448f-a226-7020a2fc39bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:06.099745 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6f65d-b46f-448f-a226-7020a2fc39bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:06.101334 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6f65d-b46f-448f-a226-7020a2fc39bc service swift] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:06.109924 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1433/2893] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:10:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:10:15.840912 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=42 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:10:16.108104 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=66 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:10:16.812686 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8fe55-1128-4c65-9d6d-f117332883ef None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:16.812982 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8fe55-1128-4c65-9d6d-f117332883ef None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:16.813291 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8fe55-1128-4c65-9d6d-f117332883ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:16.832188 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-4db8fe55-1128-4c65-9d6d-f117332883ef None None] MFA Rules not processed for user `a52eccb2ea4b4920863f4e74e75296ab`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:10:16.833598 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4db8fe55-1128-4c65-9d6d-f117332883ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:16.841996 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1461/2894] 10.222.0.81 () {58 vars in 957 bytes} [Thu Jun 16 19:10:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:10:17.094015 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6f65d-b46f-448f-a226-7020a2fc39bc service swift] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:17.095428 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-22d6f65d-b46f-448f-a226-7020a2fc39bc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:17.152533 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6f65d-b46f-448f-a226-7020a2fc39bc service swift] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:17.156130 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699fd6f-4d6f-47ea-82d4-498850c8e43a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:17.161462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699fd6f-4d6f-47ea-82d4-498850c8e43a service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:17.161462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699fd6f-4d6f-47ea-82d4-498850c8e43a service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:17.161462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699fd6f-4d6f-47ea-82d4-498850c8e43a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:17.161462 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-b699fd6f-4d6f-47ea-82d4-498850c8e43a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:17.168673 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699fd6f-4d6f-47ea-82d4-498850c8e43a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:17.170184 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699fd6f-4d6f-47ea-82d4-498850c8e43a service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:17.178147 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1434/2895] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:10:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:10:20.318081 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c0676-1f5a-486f-b980-5523da51b788 None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:20.318081 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c0676-1f5a-486f-b980-5523da51b788 None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:20.318081 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c0676-1f5a-486f-b980-5523da51b788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:20.335641 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-6f1c0676-1f5a-486f-b980-5523da51b788 None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:10:20.336953 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-6f1c0676-1f5a-486f-b980-5523da51b788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:20.349475 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1462/2896] 10.222.0.81 () {58 vars in 1009 bytes} [Thu Jun 16 19:10:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:10:20.365433 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699fd6f-4d6f-47ea-82d4-498850c8e43a service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:20.377993 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699fd6f-4d6f-47ea-82d4-498850c8e43a service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:20.383464 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bee24-a7c2-4ed5-b58b-cfd39851c08a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:20.384112 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bee24-a7c2-4ed5-b58b-cfd39851c08a service swift] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:20.384390 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bee24-a7c2-4ed5-b58b-cfd39851c08a service swift] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:20.384665 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bee24-a7c2-4ed5-b58b-cfd39851c08a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:20.393320 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bee24-a7c2-4ed5-b58b-cfd39851c08a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:20.394861 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bee24-a7c2-4ed5-b58b-cfd39851c08a service swift] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:20.403876 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1435/2897] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:10:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:10:30.330539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=6 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:10:30.399758 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=32 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:10:48.452852 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c0676-1f5a-486f-b980-5523da51b788 None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:48.462178 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c0676-1f5a-486f-b980-5523da51b788 None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:48.465664 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7cbf8-d14c-49dc-8040-4069c718d85f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:48.466539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7cbf8-d14c-49dc-8040-4069c718d85f service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:48.466539 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7cbf8-d14c-49dc-8040-4069c718d85f service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:48.466956 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7cbf8-d14c-49dc-8040-4069c718d85f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:48.475812 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7cbf8-d14c-49dc-8040-4069c718d85f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:48.477221 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf7cbf8-d14c-49dc-8040-4069c718d85f service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:48.487117 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1463/2898] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:10:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:10:50.578647 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bee24-a7c2-4ed5-b58b-cfd39851c08a service swift] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:10:50.580165 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-2a0bee24-a7c2-4ed5-b58b-cfd39851c08a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:50.679055 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bee24-a7c2-4ed5-b58b-cfd39851c08a service swift] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:10:50.685012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe00ec-a160-4750-a470-cef8d4d0e4a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:10:50.685708 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe00ec-a160-4750-a470-cef8d4d0e4a2 service nova] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:10:50.685995 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe00ec-a160-4750-a470-cef8d4d0e4a2 service nova] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:10:50.686270 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe00ec-a160-4750-a470-cef8d4d0e4a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:10:50.688228 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-91fe00ec-a160-4750-a470-cef8d4d0e4a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:10:50.702008 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe00ec-a160-4750-a470-cef8d4d0e4a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:10:50.712696 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe00ec-a160-4750-a470-cef8d4d0e4a2 service nova] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:10:50.715161 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1436/2899] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:10:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:10:58.482478 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=6 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:11:00.710971 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=43 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:11:11.158849 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7cbf8-d14c-49dc-8040-4069c718d85f service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:11:11.167502 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7cbf8-d14c-49dc-8040-4069c718d85f service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:11:11.171779 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7d51b-4fc7-481d-b5a8-f8678206104c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:11:11.172418 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7d51b-4fc7-481d-b5a8-f8678206104c service neutron] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:11:11.172705 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7d51b-4fc7-481d-b5a8-f8678206104c service neutron] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:11:11.172980 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7d51b-4fc7-481d-b5a8-f8678206104c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:11:11.174782 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-25a7d51b-4fc7-481d-b5a8-f8678206104c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:11:11.188223 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7d51b-4fc7-481d-b5a8-f8678206104c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:11:11.189682 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7d51b-4fc7-481d-b5a8-f8678206104c service neutron] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:11:11.202137 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1464/2900] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:11:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:11:12.481873 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe00ec-a160-4750-a470-cef8d4d0e4a2 service nova] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:11:12.491742 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe00ec-a160-4750-a470-cef8d4d0e4a2 service nova] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:11:12.495327 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cccf9a-1221-49b6-b62a-79f081c378ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:11:12.495959 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cccf9a-1221-49b6-b62a-79f081c378ed service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:11:12.496243 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cccf9a-1221-49b6-b62a-79f081c378ed service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:11:12.496517 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cccf9a-1221-49b6-b62a-79f081c378ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:11:12.499159 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-54cccf9a-1221-49b6-b62a-79f081c378ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:11:12.512217 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cccf9a-1221-49b6-b62a-79f081c378ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:11:12.513704 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cccf9a-1221-49b6-b62a-79f081c378ed service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:11:12.524483 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1437/2901] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:11:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:11:21.196955 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=6 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:11:22.519783 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=6 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:11:56.220600 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7d51b-4fc7-481d-b5a8-f8678206104c service neutron] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:11:56.277009 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7d51b-4fc7-481d-b5a8-f8678206104c service neutron] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:11:56.284574 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04416161-ec30-4a7d-870f-0067b9b0eb57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '251ed026ce8046e98188066bf0bc40b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:11:56.285653 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416161-ec30-4a7d-870f-0067b9b0eb57 service nova] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:11:56.286169 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416161-ec30-4a7d-870f-0067b9b0eb57 service nova] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:11:56.287060 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04416161-ec30-4a7d-870f-0067b9b0eb57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:11:56.289415 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-04416161-ec30-4a7d-870f-0067b9b0eb57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:11:56.315595 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04416161-ec30-4a7d-870f-0067b9b0eb57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:11:56.317303 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04416161-ec30-4a7d-870f-0067b9b0eb57 service nova] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:11:56.339862 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1465/2902] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:11:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:11:56.421872 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cccf9a-1221-49b6-b62a-79f081c378ed service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:11:56.432448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cccf9a-1221-49b6-b62a-79f081c378ed service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:11:56.438079 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68402dd6-bd70-4faa-bac1-165ac6c08127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4e05d75cd1142e18d7518b78edd88fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:11:56.438914 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68402dd6-bd70-4faa-bac1-165ac6c08127 service neutron] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:11:56.439325 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68402dd6-bd70-4faa-bac1-165ac6c08127 service neutron] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:11:56.439697 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68402dd6-bd70-4faa-bac1-165ac6c08127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:11:56.441815 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-68402dd6-bd70-4faa-bac1-165ac6c08127 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:11:56.453771 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68402dd6-bd70-4faa-bac1-165ac6c08127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:11:56.457020 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68402dd6-bd70-4faa-bac1-165ac6c08127 service neutron] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:11:56.468703 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1438/2903] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:11:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:12:04.368570 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c3814-7879-4490-b872-793acd14180f None None] REQUEST_METHOD: `POST` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:04.369086 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c3814-7879-4490-b872-793acd14180f None None] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:04.369202 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67c3814-7879-4490-b872-793acd14180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:04.386647 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.auth.core [None req-f67c3814-7879-4490-b872-793acd14180f None None] MFA Rules not processed for user `0d78b2dc673341dba7b8cba588471c46`. Rule list: `[]` (Enabled: `True`). {{(pid=92270) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 19:12:04.387854 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-f67c3814-7879-4490-b872-793acd14180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:04.395375 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1466/2904] 10.222.0.81 () {58 vars in 967 bytes} [Thu Jun 16 19:12:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 19:12:04.434784 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68402dd6-bd70-4faa-bac1-165ac6c08127 service neutron] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:04.446780 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68402dd6-bd70-4faa-bac1-165ac6c08127 service neutron] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:04.451654 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f9f86-ec18-4cc5-82ca-07cf417805f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b90f65f1f04b416286142c081c92a02d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:04.452288 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f9f86-ec18-4cc5-82ca-07cf417805f7 service swift] REQUEST_METHOD: `GET` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:04.452575 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f9f86-ec18-4cc5-82ca-07cf417805f7 service swift] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:04.453384 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0f9f86-ec18-4cc5-82ca-07cf417805f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:04.461352 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f9f86-ec18-4cc5-82ca-07cf417805f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:04.462873 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0f9f86-ec18-4cc5-82ca-07cf417805f7 service swift] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:04.471415 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1439/2905] 10.222.0.81 () {60 vars in 1384 bytes} [Thu Jun 16 19:12:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 19:12:12.382242 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c3814-7879-4490-b872-793acd14180f None None] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:12.445414 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67c3814-7879-4490-b872-793acd14180f None None] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:12.450018 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:12.450919 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:12.451206 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:12.451479 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] PATH_INFO: `/v3/users/a52eccb2ea4b4920863f4e74e75296ab` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:12.452651 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a52eccb2ea4b4920863f4e74e75296ab)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:12.454607 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:12.626329 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a52eccb2ea4b4920863f4e74e75296ab'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:12.626903 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a52eccb2ea4b4920863f4e74e75296ab'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:12.633791 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a52eccb2ea4b4920863f4e74e75296ab'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:12.634419 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a52eccb2ea4b4920863f4e74e75296ab'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:12.646648 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a52eccb2ea4b4920863f4e74e75296ab'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:12.720403 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1467/2906] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:12:12 2022] DELETE /identity/v3/users/a52eccb2ea4b4920863f4e74e75296ab => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:12.989608 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f9f86-ec18-4cc5-82ca-07cf417805f7 service swift] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:12.999735 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0f9f86-ec18-4cc5-82ca-07cf417805f7 service swift] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:13.058225 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:13.059082 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:13.059365 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:13.059639 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] PATH_INFO: `/v3/projects/434965eee025410ca595f90a8bd80ee5` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:13.060812 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=434965eee025410ca595f90a8bd80ee5)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:13.062782 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:13.140142 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '434965eee025410ca595f90a8bd80ee5', 'request_id': 'req-8efdb6f0-2ece-45f3-b736-07759795a1aa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:13.197841 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '434965eee025410ca595f90a8bd80ee5', 'request_id': 'req-8efdb6f0-2ece-45f3-b736-07759795a1aa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:13.203465 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '434965eee025410ca595f90a8bd80ee5', 'request_id': 'req-8efdb6f0-2ece-45f3-b736-07759795a1aa'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:13.214370 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] The token cache is being invalidate because project 434965eee025410ca595f90a8bd80ee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:12:13.214653 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:13.215448 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:13.217012 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1440/2907] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:12:12 2022] DELETE /identity/v3/projects/434965eee025410ca595f90a8bd80ee5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:13.221639 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:13.223080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:13.236542 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45d916-db14-48eb-8311-5f6ab98698e2 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:13.292080 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:13.292961 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:13.293259 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:13.293550 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] PATH_INFO: `/v3/users/d95fc270ea7e4998b52870073396e366` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:13.294739 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d95fc270ea7e4998b52870073396e366)` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:13.297385 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:13.299501 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:13.485960 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd95fc270ea7e4998b52870073396e366'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:13.486688 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd95fc270ea7e4998b52870073396e366'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:13.493363 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd95fc270ea7e4998b52870073396e366'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:13.493998 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd95fc270ea7e4998b52870073396e366'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:13.505833 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.notifications [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd95fc270ea7e4998b52870073396e366'} {{(pid=92270) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:13.595050 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1468/2908] 10.222.0.81 () {58 vars in 1223 bytes} [Thu Jun 16 19:12:13 2022] DELETE /identity/v3/users/d95fc270ea7e4998b52870073396e366 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:13.850998 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] Authenticating user token {{(pid=92269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:12:13.852460 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.fernet_utils [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:12:13.866172 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efdb6f0-2ece-45f3-b736-07759795a1aa admin admin] Validating token access rules against request {{(pid=92269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:12:13.923434 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88fee257b59d48f2b418ffa9e86b6f8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b167122bd2b4c3cba143eecf3576355', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:12:13.924300 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:12:13.924588 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:12:13.924989 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] PATH_INFO: `/v3/projects/0fdb52841fa6484a9f3e0b8f5a1335b0` {{(pid=92269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:12:13.926322 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fdb52841fa6484a9f3e0b8f5a1335b0)` {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:12:13.928235 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] RBAC: Authorization granted {{(pid=92269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:12:13.934520 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:14.026452 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fdb52841fa6484a9f3e0b8f5a1335b0', 'request_id': 'req-9b2a593b-b7de-4719-b611-6bba265853b2'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:14.123620 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fdb52841fa6484a9f3e0b8f5a1335b0', 'request_id': 'req-9b2a593b-b7de-4719-b611-6bba265853b2'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:14.130144 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fdb52841fa6484a9f3e0b8f5a1335b0', 'request_id': 'req-9b2a593b-b7de-4719-b611-6bba265853b2'} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:14.141127 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] The token cache is being invalidate because project 0fdb52841fa6484a9f3e0b8f5a1335b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 19:12:14.141444 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:14.142183 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG keystone.notifications [None req-9b2a593b-b7de-4719-b611-6bba265853b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 19:12:14.143796 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: [pid: 92269|app: 0|req: 1441/2909] 10.222.0.81 () {58 vars in 1232 bytes} [Thu Jun 16 19:12:13 2022] DELETE /identity/v3/projects/0fdb52841fa6484a9f3e0b8f5a1335b0 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 19:12:23.590790 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:12:24.138085 dv-cinder-846171-1-iscsi devstack@keystone.service[92269]: DEBUG dbcounter [-] [92269] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 19:14:11.553711 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] Authenticating user token {{(pid=92270) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 19:14:11.555182 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:14:11.623752 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977fb28-730b-4872-842c-9c4c5b30e992 admin admin] Validating token access rules against request {{(pid=92270) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 19:14:11.683064 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fac50d-5e7e-401a-95f0-c84eac007456 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20c0fd976e684356b6143dc33fb7b4af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dc7e521e91b4468dbe5b02121e21c8c1', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92270) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 19:14:11.683719 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fac50d-5e7e-401a-95f0-c84eac007456 service placement] REQUEST_METHOD: `GET` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 19:14:11.684004 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fac50d-5e7e-401a-95f0-c84eac007456 service placement] SCRIPT_NAME: `/identity` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 19:14:11.684283 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fac50d-5e7e-401a-95f0-c84eac007456 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92270) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 19:14:11.686124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.fernet_utils [None req-55fac50d-5e7e-401a-95f0-c84eac007456 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92270) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 19:14:11.697829 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fac50d-5e7e-401a-95f0-c84eac007456 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 19:14:11.699398 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fac50d-5e7e-401a-95f0-c84eac007456 service placement] RBAC: Authorization granted {{(pid=92270) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 19:14:11.709124 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: [pid: 92270|app: 0|req: 1469/2910] 10.222.0.81 () {60 vars in 1365 bytes} [Thu Jun 16 19:14:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 19:14:21.705061 dv-cinder-846171-1-iscsi devstack@keystone.service[92270]: DEBUG dbcounter [-] [92270] Writing DB stats keystone:SELECT=58 {{(pid=92270) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}